-- Logs begin at Fri 2022-09-23 00:26:38 UTC, end at Fri 2022-09-23 01:53:08 UTC. -- Sep 23 01:04:18.098753 dv-cinder-849143-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 23 01:04:18.109387 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 23 01:04:18 2022] *** Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: nodename: dv-cinder-849143-7-iscsi Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: machine: x86_64 Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: clock source: unix Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: pcre jit disabled Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: detected number of CPU cores: 4 Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: current working directory: / Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: detected binary path: /usr/bin/uwsgi-core Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: your processes number limit is 47680 Sep 23 01:04:18.111417 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: your memory page size is 4096 bytes Sep 23 01:04:18.112186 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: detected max file descriptor number: 2048 Sep 23 01:04:18.112186 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: lock engine: pthread robust mutexes Sep 23 01:04:18.112186 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: thunder lock: enabled Sep 23 01:04:18.112186 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 23 01:04:18.112186 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 23 01:04:18.153531 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: Python main interpreter initialized at 0x564cfc5829c0 Sep 23 01:04:18.153531 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: python threads support enabled Sep 23 01:04:18.153531 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: your server socket listen backlog is limited to 100 connections Sep 23 01:04:18.153531 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: your mercy for graceful operations on workers is 90 seconds Sep 23 01:04:18.153937 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: mapped 403077 bytes (393 KB) for 2 cores Sep 23 01:04:18.153937 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: *** Operational MODE: preforking *** Sep 23 01:04:18.154232 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: *** uWSGI is running in multiple interpreter mode *** Sep 23 01:04:18.154232 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: spawned uWSGI master process (pid: 81948) Sep 23 01:04:18.154443 dv-cinder-849143-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 23 01:04:18.159118 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: spawned uWSGI worker 1 (pid: 81960, cores: 1) Sep 23 01:04:18.159118 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: spawned uWSGI worker 2 (pid: 81961, cores: 1) Sep 23 01:04:18.159118 dv-cinder-849143-7-iscsi devstack@keystone.service[81948]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 23 01:04:22.664927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 01:04:22.664927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: from cryptography.utils import int_from_bytes Sep 23 01:04:22.665927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 01:04:22.665927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: from cryptography.utils import int_from_bytes Sep 23 01:04:22.673810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 01:04:22.673810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: from cryptography.utils import int_from_bytes Sep 23 01:04:22.674674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 23 01:04:22.674674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: from cryptography.utils import int_from_bytes Sep 23 01:04:22.717077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 23 01:04:22.717324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 23 01:04:22.717458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] command line args: [] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 23 01:04:22.717700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 23 01:04:22.717852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ================================================================================ {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 23 01:04:22.718117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] admin_token = **** {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.718414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] config_dir = [] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.718608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.718786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] config_source = [] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.718966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] debug = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.719152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.719594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.719780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.719940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.720203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.720436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] list_limit = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.720610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] log_config_append = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.720771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.720939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] log_dir = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.721100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] log_file = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.721325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.721571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.721814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.722096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.722279 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.722438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.722613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.722856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.723091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.723310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.723542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.723797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.724045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.724217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 23 01:04:22.724372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 23 01:04:22.724511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.724651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] command line args: [] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 23 01:04:22.725486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.725632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 23 01:04:22.725632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ================================================================================ {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 23 01:04:22.725951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.726082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] admin_token = **** {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.726277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] publish_errors = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.726417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] config_dir = [] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.726617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.726756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.726949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.727075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] config_source = [] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.727256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.727393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] debug = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.727621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.727753 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.727989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.728173 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.728173 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.728480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] standard_threads = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.728480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.728768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.728959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.729099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.729227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.729227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] use_journal = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.729541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] list_limit = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.729690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] use_json = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.729872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] log_config_append = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.730019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] use_stderr = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.730156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.730359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] use_syslog = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.730495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] log_dir = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.730622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.730757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] log_file = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.730880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.731010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.731153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.731289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.731289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.731531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.731531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.731949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.731949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.732181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.732340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.732476 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.732620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.732754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.732881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.733020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.733148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.733291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.733417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.733542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.733674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.733674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.733900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.733900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.734249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.734381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.734571 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.734708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.734834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.734974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.734974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.735204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.735330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.735475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] publish_errors = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.735670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.735800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.735933 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.736126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.736126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.736386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.736521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.736649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.736774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.736909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.737038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] standard_threads = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.737177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.737310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.737441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.737580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.737706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.737843 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.737986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.738125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] use_journal = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.738125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] use_json = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.738359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.738482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] use_stderr = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.738617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.738758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] use_syslog = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.738893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.739020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 23 01:04:22.739152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.739287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.739418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.739593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.739732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.739859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.739985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.740108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.740282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.740516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.740643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.740776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.740899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.741032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.741157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.741300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.741424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.741558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.741693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.741825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.741950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.742086 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.742214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.742350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.742485 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.742609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.742746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.742885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.743023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.743155 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.743283 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.743283 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.743542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.743748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.743875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.744002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.744124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.744248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.744433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.744648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.744825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.744992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.745166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.745330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.745514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.745682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.745870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.745998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.746116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.746236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.746381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.746509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.746637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.746785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.746926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.747066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.747194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.747339 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.747468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.747630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.747759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.747930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.748062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.748261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.748438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.748621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.748772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.748964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.749099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.749278 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.749428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.749614 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.749738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.749922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.750105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.750299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.750428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.750556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.750732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.750917 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.751047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.751193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.751322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.751450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.751635 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.751767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.751894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.752023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.752170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.752299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.752438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.752565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.752768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.752909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.753043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.753182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.753310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.753435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.753561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.753701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.753830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.753959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.754083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.754209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.754209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.754456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.754582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.754708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.754838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.754966 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.755091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.755329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.755474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.755627 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.755773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.756470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.756737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.756870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.757013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.757143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.757288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.757416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.757544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.757670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.757928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.758145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.758277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.758406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.758549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.758676 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.758803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.758944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.759087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.759214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.759341 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.759469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.759618 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.759765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.759902 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.760042 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.760172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.760300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.760427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.760571 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.760699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.760699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.760929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.760929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.761163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.761304 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.761430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.761566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.761705 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.761835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.761959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] auth.external = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.761959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.762213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.762213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.762454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.762639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.762835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.762957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.763093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] auth.password = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.763093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] auth.token = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.763334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.763464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.763728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.763870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.764006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.764125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.764255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.764383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.764534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.764534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.764770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.764892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.765018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.765018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.765273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.765418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] credential.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.765545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.765672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.765802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.765931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.766202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.766202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.766474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.766615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.766745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.766882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.767017 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.767147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.767291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.767408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.767522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.767662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.767775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.767886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.768014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.768127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.768238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.768348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.768479 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.768619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.768768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.768906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.769033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.769161 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.769302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.769422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.769549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.769674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.769803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] federation.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.769930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.770067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.770193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.770335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.770466 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] auth.external = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.770593 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.770719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.770848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.770978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.771215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.771347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.771486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.772311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] auth.password = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.772444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.772576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] auth.token = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.772727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.772855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.772992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.773121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.773251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.773380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.773521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.773653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.773782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.773782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.774016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.774145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.774288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.774420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.774548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.774684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.774813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] credential.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.774943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.775087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.775214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.775351 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.775480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.775636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.775781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.775907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.776020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.776131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.776260 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.776385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.776514 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.776655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.776782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.776913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.777040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.777168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.777310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.777440 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.777568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.777694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.777822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.777940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.778071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.778181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.778315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.778446 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.778575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.778702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.778849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.778981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] federation.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.779096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.779215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.779342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.779468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.779683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.779683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.779923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.780053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.780182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.780308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.780447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.780584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.780714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.780842 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.780972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.781104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.781244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.781372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.781499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.781629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.781757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.781899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.782042 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.782173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.782297 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.782424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.782561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.782689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.782830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.782959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.783087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.783216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.783344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.783470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.783647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.783777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.783915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.784048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.784176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.784305 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.784449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.784581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.784707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.784707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.784934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.785048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.785175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.785175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.785372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.785492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.785620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.785751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.785890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.786016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.786143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.786272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.786399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.786525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.786665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.786797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.786926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.787062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.787192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.787320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.787463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.787615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.787755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.787884 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.788013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.788159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.788274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.788405 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.788544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.788672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.788799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.788941 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.789073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.789203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.789331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.789459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.789586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.789743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.789872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.789985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.790115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.790243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.790374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.790517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.790645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.790775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.790912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.791041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.791169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.791307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.791437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.791653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.791784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.791925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.792058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.792198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.792326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.792443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.792562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.792684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.792811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.792952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.793081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.793211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.793340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.793469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.793595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.793752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.793891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.794021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.794153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.794280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.794407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.794547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.794677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.794807 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.794938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.795068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.795068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.795314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.795442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.795603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.795733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.795847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.795973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.796096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.796223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.796352 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.796492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] resource.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.796619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.796745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.796989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.797121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.797249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.797377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.797506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.797636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.797776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.797906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.798035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.798161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.798286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.798286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.798508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.798634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.798762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.798762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.798991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.799117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] role.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.799259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.799389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] role.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.799507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.799734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.799866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.799993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.800136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.800265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.800393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.800521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.800651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.800780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.800922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.801050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.801163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.801304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.801431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.801559 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.801700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.801829 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.801957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.802086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.802215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.802342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.802484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.802613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.802742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.802871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.803000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.803142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.803274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.803401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.803528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.803749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.803878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.804023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.804153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.804283 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.804469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.804600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.804730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.804872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.805003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.805130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.805267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.805499 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.805632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.805775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.805904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] resource.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.806034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.806161 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.806289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.806447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.806592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.806723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.806836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.806958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.807088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.807217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.807359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.807488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.807785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.807954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.808120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.808284 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.808475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.808651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.808817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.808984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] role.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.809151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] token.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.809313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] role.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.809483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.809651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.809818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.809994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.810157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.810320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.810503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.810669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.810832 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.811002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.811167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.811330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.811517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.811697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.811852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.812006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.812166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.812335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.812514 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.812674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.812831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.813002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.813163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.813326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.813504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.813685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.813853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.814020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.814183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.814183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.814517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.814687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.814844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.815004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.connection = **** {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.815157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.815373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.815527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.815732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.815882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.816016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.816143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.816265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.816381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.816508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.816508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.816750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.816750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.816951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.817069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.817202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.817323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.817447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.817571 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.817696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.817818 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.817955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.818077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.818198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.818321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.819176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.819299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.819423 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.819643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.819771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.819896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] token.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81960) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 23 01:04:22.820778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.820778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.821732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.821732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.821732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.821732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.821732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.821732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.821732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.821732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.821732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.connection = **** {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.822814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.822814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.822814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.822814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.822814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.822814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.822814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.822814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.822814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.824065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.824065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.824065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.824065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.824065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.824065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.824065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.824065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.824065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 23 01:04:22.825370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81961) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 23 01:04:22.837824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.838120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.838265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.842277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.842491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.842727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.847203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.851845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.876648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.876907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.879452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.879743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.936128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.936764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.988951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.989253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.992193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:22.992521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.157344 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.157551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.157627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.157777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.157873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.158015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.158197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.158343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.158480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.158634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.158794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.158932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.183645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.183901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.184066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.184212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.184358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.184573 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.184717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.184867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.185006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.185144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.185281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.185415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.185568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.185568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.197533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.197738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.211608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.211851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.212079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.212303 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.212520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.212735 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.212952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81960) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.214071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.214479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.214885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.215284 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.215696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.216077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.216470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81961) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 23 01:04:23.230911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.231165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 01:04:23.231459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.231681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 01:04:23.231958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.232224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.232487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.232747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.233384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.233597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 01:04:23.234146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.234494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 23 01:04:23.234817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.235094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.235371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.235665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.242591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.242887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.243161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.243454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.251091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.251506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.251975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.252573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.256806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.261838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.261994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.262436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.262931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.263424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.263949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.264403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.264874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.264981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.265167 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.265359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.265460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.265741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.265830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.266034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.266307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.266428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.266581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.266803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.266932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.267126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.267409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.267708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.292009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.293627 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.293921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.298877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.299169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.299581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.301226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.302766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.303092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.306474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.306784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.307837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.308136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.308425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.312868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.313175 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.313470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.313765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.314051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.314333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.314611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.314818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.315028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.315324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.321414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.321726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.322022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.322313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.322600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.322881 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.323158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.323433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.331034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.331310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.335854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.339077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.339432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.339650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.339938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.343977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.344285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.347312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.347621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.347834 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.351299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.351593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.351892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.352197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.352447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.352677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.352880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.355627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.358891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.359197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.359483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.359801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.360102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.360398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.369621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.369934 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.370213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.370489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.376840 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.377152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.377424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.377582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.377726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.379164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.380761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.381187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.384003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.384313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.384617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.384773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.384901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.385306 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.386262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.394400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.394400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.394400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.394400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.394400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.394400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.394907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.397650 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.397896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.398106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.398360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.398719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.399005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.399291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.404023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.404319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.404601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.404882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.405328 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.405565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.405844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.415639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.418640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.421754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.421973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.423213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.424722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.424897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.425050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.425410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.427814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.429300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.430781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.431063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.431470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.432141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.433696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.436624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.437030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.437321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.437605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.437838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.439375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.442304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.442710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.443008 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.443288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.445889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.446202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.446499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.451819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.452118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.452360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.453613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81961) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.453934 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.454281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.454576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.455023 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.455311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.455619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.455895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.456187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.456467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.459838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.459838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.459838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.459838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81961) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.460710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81960) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 23 01:04:23.460710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.460710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.461430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.461430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.461430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.462011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.462011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.462416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.462416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.462768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.463011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.463321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.463658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81960) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 23 01:04:23.494295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564cfc5829c0 pid: 81961 (default app) Sep 23 01:04:23.497951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9410ca-1c2b-4a86-b904-3f9c9f0f35a0 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:23.498239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9410ca-1c2b-4a86-b904-3f9c9f0f35a0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:23.498710 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9410ca-1c2b-4a86-b904-3f9c9f0f35a0 None None] PATH_INFO: `/v3` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:23.498866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564cfc5829c0 pid: 81960 (default app) Sep 23 01:04:23.503029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1/1] 10.222.0.64 () {50 vars in 791 bytes} [Fri Sep 23 01:04:23 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:32.035389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907672f-3c73-4844-b09f-540986cea502 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:32.036209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907672f-3c73-4844-b09f-540986cea502 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:32.036209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907672f-3c73-4844-b09f-540986cea502 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:32.040527 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1/2] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:32.047770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:32.048342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:32.048342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:32.120616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: INFO dbcounter [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] Registered counter for database keystone Sep 23 01:04:32.142756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG oslo_db.sqlalchemy.engines [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81961) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 23 01:04:32.149355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writer thread running {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 23 01:04:32.371653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG passlib.handlers.bcrypt [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81961) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 23 01:04:32.371970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG passlib.handlers.bcrypt [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81961) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 23 01:04:32.431224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:32.432915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:32.566666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 2/3] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 23 01:04:32.877063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907672f-3c73-4844-b09f-540986cea502 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:32.951622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: INFO dbcounter [None req-f907672f-3c73-4844-b09f-540986cea502 None None] Registered counter for database keystone Sep 23 01:04:32.972424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG oslo_db.sqlalchemy.engines [None req-f907672f-3c73-4844-b09f-540986cea502 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81960) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 23 01:04:32.980539 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writer thread running {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 23 01:04:33.107746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907672f-3c73-4844-b09f-540986cea502 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:33.112446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22ac94-3e43-4f76-8526-4ea0c35959b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:33.113102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22ac94-3e43-4f76-8526-4ea0c35959b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:33.113387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22ac94-3e43-4f76-8526-4ea0c35959b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:33.113667 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22ac94-3e43-4f76-8526-4ea0c35959b8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:33.123249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22ac94-3e43-4f76-8526-4ea0c35959b8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:33.176407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ef22ac94-3e43-4f76-8526-4ea0c35959b8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 23 01:04:33.176760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22ac94-3e43-4f76-8526-4ea0c35959b8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:33.187592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 2/4] 10.222.0.64 () {58 vars in 1324 bytes} [Fri Sep 23 01:04:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:04:33.194013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:33.212401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b34dc77-bd02-4f02-8742-d5d889c4b74f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:33.216845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2a979-3e7a-47fb-84ce-808d2029cbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:33.217989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a979-3e7a-47fb-84ce-808d2029cbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:33.218250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a979-3e7a-47fb-84ce-808d2029cbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:33.218522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2a979-3e7a-47fb-84ce-808d2029cbd0 admin admin] PATH_INFO: `/v3/projects/9c4e8ab21bdd4e5ebc7ac91f1638bec3` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:33.220171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2a979-3e7a-47fb-84ce-808d2029cbd0 admin admin] RBAC: Authorizing `identity:get_project(project_id=9c4e8ab21bdd4e5ebc7ac91f1638bec3)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:33.273840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3ec2a979-3e7a-47fb-84ce-808d2029cbd0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 23 01:04:33.274182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2a979-3e7a-47fb-84ce-808d2029cbd0 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:33.279262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 3/5] 10.222.0.64 () {56 vars in 1207 bytes} [Fri Sep 23 01:04:33 2022] GET /identity/v3/projects/9c4e8ab21bdd4e5ebc7ac91f1638bec3 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:04:35.455034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37ba19-2fdc-40a1-bbc5-7a3c63407217 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:35.455641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37ba19-2fdc-40a1-bbc5-7a3c63407217 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:35.455641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37ba19-2fdc-40a1-bbc5-7a3c63407217 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:35.462805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 3/6] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:35.469352 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042ef9c-71e2-446d-9c91-538730e1bee2 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:35.469854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042ef9c-71e2-446d-9c91-538730e1bee2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:35.469982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042ef9c-71e2-446d-9c91-538730e1bee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:35.492908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e042ef9c-71e2-446d-9c91-538730e1bee2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:35.494516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e042ef9c-71e2-446d-9c91-538730e1bee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:35.503770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 4/7] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 23 01:04:35.785529 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37ba19-2fdc-40a1-bbc5-7a3c63407217 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:35.799939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37ba19-2fdc-40a1-bbc5-7a3c63407217 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:35.806127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d3f62-21fc-4058-a1c4-ff5c444dcb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:35.807219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d3f62-21fc-4058-a1c4-ff5c444dcb6c admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:35.807733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d3f62-21fc-4058-a1c4-ff5c444dcb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:35.808064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d3f62-21fc-4058-a1c4-ff5c444dcb6c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:35.820718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d3f62-21fc-4058-a1c4-ff5c444dcb6c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:35.823461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d3f62-21fc-4058-a1c4-ff5c444dcb6c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:35.834143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 4/8] 10.222.0.64 () {58 vars in 1324 bytes} [Fri Sep 23 01:04:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:04:35.840372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042ef9c-71e2-446d-9c91-538730e1bee2 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:35.850737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042ef9c-71e2-446d-9c91-538730e1bee2 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:35.855447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d17e3c-c902-43c1-ba44-7036d349eac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:35.856136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d17e3c-c902-43c1-ba44-7036d349eac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:35.856426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d17e3c-c902-43c1-ba44-7036d349eac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:35.856707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d17e3c-c902-43c1-ba44-7036d349eac9 admin admin] PATH_INFO: `/v3/users/efae8da899ef465d9872282b2bb91fef` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:35.858112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d17e3c-c902-43c1-ba44-7036d349eac9 admin admin] RBAC: Authorizing `identity:get_user(user_id=efae8da899ef465d9872282b2bb91fef)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:35.860108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d17e3c-c902-43c1-ba44-7036d349eac9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:35.861835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 5/9] 10.222.0.64 () {56 vars in 1198 bytes} [Fri Sep 23 01:04:35 2022] GET /identity/v3/users/efae8da899ef465d9872282b2bb91fef => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:38.466912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add37ca7-5c14-4d95-a7ad-8a0c9f9941fe None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:38.467752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add37ca7-5c14-4d95-a7ad-8a0c9f9941fe None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:38.467752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add37ca7-5c14-4d95-a7ad-8a0c9f9941fe None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:38.469359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 5/10] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:38.476135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1932d-2c65-4e6f-9089-ea1b09482fb8 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:38.476909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1932d-2c65-4e6f-9089-ea1b09482fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:38.477114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1932d-2c65-4e6f-9089-ea1b09482fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:38.499851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b3b1932d-2c65-4e6f-9089-ea1b09482fb8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:38.501158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b3b1932d-2c65-4e6f-9089-ea1b09482fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:38.606686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 6/11] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:38.884345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add37ca7-5c14-4d95-a7ad-8a0c9f9941fe None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:39.069258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add37ca7-5c14-4d95-a7ad-8a0c9f9941fe None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:39.113548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478512c4-ea16-4ce8-aa6b-61194ca256db None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:39.113773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478512c4-ea16-4ce8-aa6b-61194ca256db None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:39.113979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478512c4-ea16-4ce8-aa6b-61194ca256db None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:39.115988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 7/12] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:39.116259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53549131-100d-4a57-aa36-6471a10a7444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:39.117340 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53549131-100d-4a57-aa36-6471a10a7444 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:39.117910 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53549131-100d-4a57-aa36-6471a10a7444 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:39.118418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53549131-100d-4a57-aa36-6471a10a7444 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:39.122058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fd5ce-ad1a-46ef-aab4-9a9ab279b389 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:39.122356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fd5ce-ad1a-46ef-aab4-9a9ab279b389 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:39.122633 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fd5ce-ad1a-46ef-aab4-9a9ab279b389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:39.153036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-191fd5ce-ad1a-46ef-aab4-9a9ab279b389 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:39.154951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-191fd5ce-ad1a-46ef-aab4-9a9ab279b389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:39.188490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:39.201382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:39.247886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53549131-100d-4a57-aa36-6471a10a7444 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:39.249441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53549131-100d-4a57-aa36-6471a10a7444 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:39.281889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 8/13] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:04:39.374806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 6/14] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:04:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:04:39.381445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fd5ce-ad1a-46ef-aab4-9a9ab279b389 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:39.566486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fd5ce-ad1a-46ef-aab4-9a9ab279b389 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:39.598375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53549131-100d-4a57-aa36-6471a10a7444 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:39.631652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222d896-22f0-4716-9420-aa81d8bae701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:39.632670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222d896-22f0-4716-9420-aa81d8bae701 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:39.633167 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222d896-22f0-4716-9420-aa81d8bae701 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:39.633653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222d896-22f0-4716-9420-aa81d8bae701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:39.653151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222d896-22f0-4716-9420-aa81d8bae701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:39.656167 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222d896-22f0-4716-9420-aa81d8bae701 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-4222d896-22f0-4716-9420-aa81d8bae701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:04:39.692077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:04:39.693492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:04:39.694745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:04:39.694745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:04:39.694745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:04:39.694745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 9/15] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:04:39.698278 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:39.703634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222d896-22f0-4716-9420-aa81d8bae701 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:39.710270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:39.729843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53549131-100d-4a57-aa36-6471a10a7444 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:39.775965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fb23f-8750-40ec-a279-ede36b564f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:39.776991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fb23f-8750-40ec-a279-ede36b564f11 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:39.777367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fb23f-8750-40ec-a279-ede36b564f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:39.777736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fb23f-8750-40ec-a279-ede36b564f11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:39.792355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fb23f-8750-40ec-a279-ede36b564f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-1b7fb23f-8750-40ec-a279-ede36b564f11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:04:39.801626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:04:39.803310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:04:39.804552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 7/16] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:04:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:04:39.809861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fb23f-8750-40ec-a279-ede36b564f11 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:39.905571 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222d896-22f0-4716-9420-aa81d8bae701 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:39.974043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5ba58-f276-4c99-a6e5-a8d1d5e2ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:39.975195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5ba58-f276-4c99-a6e5-a8d1d5e2ff8d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:39.975797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5ba58-f276-4c99-a6e5-a8d1d5e2ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:39.976321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5ba58-f276-4c99-a6e5-a8d1d5e2ff8d None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:39.977634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5ba58-f276-4c99-a6e5-a8d1d5e2ff8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:04:39.978214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5ba58-f276-4c99-a6e5-a8d1d5e2ff8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:39.981197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5ba58-f276-4c99-a6e5-a8d1d5e2ff8d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:39.982448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fb23f-8750-40ec-a279-ede36b564f11 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:40.011602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 10/17] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:40.042380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3402ce7-4108-41f3-9b2a-c0c96c0243bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:40.043145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3402ce7-4108-41f3-9b2a-c0c96c0243bf None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:40.043435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3402ce7-4108-41f3-9b2a-c0c96c0243bf None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:40.043802 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3402ce7-4108-41f3-9b2a-c0c96c0243bf None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:40.044528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3402ce7-4108-41f3-9b2a-c0c96c0243bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:04:40.044848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3402ce7-4108-41f3-9b2a-c0c96c0243bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:40.046364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3402ce7-4108-41f3-9b2a-c0c96c0243bf None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:40.048590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:40.063723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 8/18] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:04:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:04:40.071800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5ba58-f276-4c99-a6e5-a8d1d5e2ff8d None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:40.203891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:40.276203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5ba58-f276-4c99-a6e5-a8d1d5e2ff8d None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:40.323110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b366d-c15e-4d3f-ba28-4e26add37f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:40.324218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b366d-c15e-4d3f-ba28-4e26add37f0d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:40.324719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b366d-c15e-4d3f-ba28-4e26add37f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:40.325203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b366d-c15e-4d3f-ba28-4e26add37f0d None admin] PATH_INFO: `/v3/users/efae8da899ef465d9872282b2bb91fef` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:40.327470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b366d-c15e-4d3f-ba28-4e26add37f0d None admin] RBAC: Authorizing `identity:get_user(user_id=efae8da899ef465d9872282b2bb91fef)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:40.330334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b366d-c15e-4d3f-ba28-4e26add37f0d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:40.334477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 11/19] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:04:40 2022] GET /identity/v3/users/efae8da899ef465d9872282b2bb91fef => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:40.341048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3402ce7-4108-41f3-9b2a-c0c96c0243bf None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:40.523737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3402ce7-4108-41f3-9b2a-c0c96c0243bf None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:40.592409 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30e933-d7dd-451d-bd4b-69f1713b154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:40.593553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30e933-d7dd-451d-bd4b-69f1713b154b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:40.593879 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30e933-d7dd-451d-bd4b-69f1713b154b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:40.594164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30e933-d7dd-451d-bd4b-69f1713b154b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:40.596856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30e933-d7dd-451d-bd4b-69f1713b154b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:40.600558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30e933-d7dd-451d-bd4b-69f1713b154b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:40.603820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 9/20] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:04:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:40.611736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b366d-c15e-4d3f-ba28-4e26add37f0d None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:40.809972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b366d-c15e-4d3f-ba28-4e26add37f0d None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:40.852457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:40.861479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf59053d-bd4d-41eb-9d14-d3fdaaca9d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:40.862252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59053d-bd4d-41eb-9d14-d3fdaaca9d21 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:40.862632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59053d-bd4d-41eb-9d14-d3fdaaca9d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:40.863270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf59053d-bd4d-41eb-9d14-d3fdaaca9d21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:40.863967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59053d-bd4d-41eb-9d14-d3fdaaca9d21 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.domain.id=default, user.id=efae8da899ef465d9872282b2bb91fef) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:04:40.865973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59053d-bd4d-41eb-9d14-d3fdaaca9d21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:40.865973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf59053d-bd4d-41eb-9d14-d3fdaaca9d21 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:40.874880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 12/21] 10.222.0.64 () {56 vars in 1322 bytes} [Fri Sep 23 01:04:40 2022] GET /identity/v3/role_assignments?user.id=efae8da899ef465d9872282b2bb91fef&scope.domain.id=default&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 233 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:04:44.380198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b2342-8d47-4b65-ac45-402ee8d48122 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:44.381040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b2342-8d47-4b65-ac45-402ee8d48122 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:44.381167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b2342-8d47-4b65-ac45-402ee8d48122 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:44.385640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 10/22] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:04:44.390824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa3f2f-b7d6-4025-85e2-e6a2d10e4ec2 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:44.391708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa3f2f-b7d6-4025-85e2-e6a2d10e4ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:44.391874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa3f2f-b7d6-4025-85e2-e6a2d10e4ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:44.412731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7a0e-4002-4866-9d7e-fa44168aa45b None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:44.413599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7a0e-4002-4866-9d7e-fa44168aa45b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:44.414358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be7a0e-4002-4866-9d7e-fa44168aa45b None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:44.417472 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 11/23] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:44.441935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-c9aa3f2f-b7d6-4025-85e2-e6a2d10e4ec2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:44.442177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e5fc-27e6-4f37-9f99-45b537edeea6 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:44.442314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e5fc-27e6-4f37-9f99-45b537edeea6 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:44.442512 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e5fc-27e6-4f37-9f99-45b537edeea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:44.444354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c9aa3f2f-b7d6-4025-85e2-e6a2d10e4ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:44.479299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG passlib.handlers.bcrypt [None req-d648e5fc-27e6-4f37-9f99-45b537edeea6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81960) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 23 01:04:44.480080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG passlib.handlers.bcrypt [None req-d648e5fc-27e6-4f37-9f99-45b537edeea6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81960) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 23 01:04:44.569574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d648e5fc-27e6-4f37-9f99-45b537edeea6 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:44.571233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d648e5fc-27e6-4f37-9f99-45b537edeea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:44.608861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 13/24] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 23 01:04:44.675041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d337a8-dcf8-481b-92ff-4b51cc0e52f2 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:44.675314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d337a8-dcf8-481b-92ff-4b51cc0e52f2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:44.675525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d337a8-dcf8-481b-92ff-4b51cc0e52f2 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:44.677198 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 14/25] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:44.687095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:44.688446 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a9059-da70-4fb2-8824-b57adc01e3ae None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:44.688739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a9059-da70-4fb2-8824-b57adc01e3ae None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:44.689017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a9059-da70-4fb2-8824-b57adc01e3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:44.715806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 12/26] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:44.720770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-4f1a9059-da70-4fb2-8824-b57adc01e3ae None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:44.722027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4f1a9059-da70-4fb2-8824-b57adc01e3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:44.835275 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 15/27] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:45.036051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ad07-1eed-498a-9e17-974bed0a2899 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:45.036523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ad07-1eed-498a-9e17-974bed0a2899 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:45.036934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ad07-1eed-498a-9e17-974bed0a2899 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:45.039894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 13/28] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:45.047359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54b78c-d3b6-4519-b285-7689cca50615 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:45.047938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54b78c-d3b6-4519-b285-7689cca50615 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:45.048385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54b78c-d3b6-4519-b285-7689cca50615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:45.076617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-db54b78c-d3b6-4519-b285-7689cca50615 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:45.078233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-db54b78c-d3b6-4519-b285-7689cca50615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:45.100651 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307ad07-1eed-498a-9e17-974bed0a2899 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:45.184151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:45.244437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 16/29] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:04:45.365740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307ad07-1eed-498a-9e17-974bed0a2899 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:45.413548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274f974-c763-4bfb-ac9d-c975e0b05b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:45.414449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274f974-c763-4bfb-ac9d-c975e0b05b3b None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:45.414449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274f974-c763-4bfb-ac9d-c975e0b05b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:45.414878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274f974-c763-4bfb-ac9d-c975e0b05b3b None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:45.415687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274f974-c763-4bfb-ac9d-c975e0b05b3b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:45.417272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a274f974-c763-4bfb-ac9d-c975e0b05b3b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:45.472389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54b78c-d3b6-4519-b285-7689cca50615 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:45.622305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54b78c-d3b6-4519-b285-7689cca50615 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:45.667211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa9a8e-ea91-44b6-a97a-767801ba0762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:45.668132 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa9a8e-ea91-44b6-a97a-767801ba0762 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:45.668665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa9a8e-ea91-44b6-a97a-767801ba0762 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:45.669180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa9a8e-ea91-44b6-a97a-767801ba0762 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:45.670348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa9a8e-ea91-44b6-a97a-767801ba0762 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:45.672344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa9a8e-ea91-44b6-a97a-767801ba0762 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:45.727968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 14/30] 10.222.0.64 () {60 vars in 1131 bytes} [Fri Sep 23 01:04:45 2022] POST /identity/v3/roles => generated 219 bytes in 628 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:04:45.729725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f16d6f-13d5-46f1-a14b-e946d31ee156 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:45.730023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f16d6f-13d5-46f1-a14b-e946d31ee156 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:45.730324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f16d6f-13d5-46f1-a14b-e946d31ee156 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:45.731891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 15/31] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:45.733006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80eff9-f273-466b-af89-0e745043cb75 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:45.733376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80eff9-f273-466b-af89-0e745043cb75 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:45.733802 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80eff9-f273-466b-af89-0e745043cb75 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:45.737907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 16/32] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:45.737907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80eff9-f273-466b-af89-0e745043cb75 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:45.822182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:45.904818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80eff9-f273-466b-af89-0e745043cb75 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:45.960841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c8fc1-6f50-4a97-bc96-bbac980a1e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:45.962428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c8fc1-6f50-4a97-bc96-bbac980a1e9b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:45.962938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c8fc1-6f50-4a97-bc96-bbac980a1e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:45.963461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c8fc1-6f50-4a97-bc96-bbac980a1e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:45.974989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 17/33] 10.222.0.64 () {60 vars in 1131 bytes} [Fri Sep 23 01:04:45 2022] POST /identity/v3/roles => generated 225 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:04:45.976895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa9a8e-ea91-44b6-a97a-767801ba0762 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:45.994418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c8fc1-6f50-4a97-bc96-bbac980a1e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:45.996600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c8fc1-6f50-4a97-bc96-bbac980a1e9b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-fe5c8fc1-6f50-4a97-bc96-bbac980a1e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:04:46.020129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:04:46.021751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:04:46.023289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:04:46.023289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:04:46.023289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:04:46.025415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 17/34] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:04:46.026677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee41f6-56fb-4659-9341-1a67cf0e6bef None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:46.027105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee41f6-56fb-4659-9341-1a67cf0e6bef None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:46.027656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee41f6-56fb-4659-9341-1a67cf0e6bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:46.055803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a6ee41f6-56fb-4659-9341-1a67cf0e6bef None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:46.058070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a6ee41f6-56fb-4659-9341-1a67cf0e6bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:46.078696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:46.151428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa9a8e-ea91-44b6-a97a-767801ba0762 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:46.157571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 18/35] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:46.158835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49824f93-14aa-4938-905f-15cbb0d6f7f8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:46.159090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49824f93-14aa-4938-905f-15cbb0d6f7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:46.159677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49824f93-14aa-4938-905f-15cbb0d6f7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:46.190817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-49824f93-14aa-4938-905f-15cbb0d6f7f8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:46.195745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-49824f93-14aa-4938-905f-15cbb0d6f7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:46.210350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba02077-2f1d-4da4-8caf-674b091f3211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:46.211844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba02077-2f1d-4da4-8caf-674b091f3211 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:46.212520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba02077-2f1d-4da4-8caf-674b091f3211 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:46.213145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba02077-2f1d-4da4-8caf-674b091f3211 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:46.233007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba02077-2f1d-4da4-8caf-674b091f3211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:04:46.235470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-1ba02077-2f1d-4da4-8caf-674b091f3211 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:04:46.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:04:46.242084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:04:46.242839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:04:46.246837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 18/36] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:04:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:04:46.248225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba02077-2f1d-4da4-8caf-674b091f3211 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:46.312118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 19/37] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:46.320418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49824f93-14aa-4938-905f-15cbb0d6f7f8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:46.444526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba02077-2f1d-4da4-8caf-674b091f3211 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:46.453515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:46.464849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49824f93-14aa-4938-905f-15cbb0d6f7f8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:46.486385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae7671-9c57-4285-8fac-8d28fd1c626a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:46.487837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae7671-9c57-4285-8fac-8d28fd1c626a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:46.488365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae7671-9c57-4285-8fac-8d28fd1c626a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:46.488876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae7671-9c57-4285-8fac-8d28fd1c626a None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:46.490388 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae7671-9c57-4285-8fac-8d28fd1c626a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:04:46.490999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae7671-9c57-4285-8fac-8d28fd1c626a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:46.492913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae7671-9c57-4285-8fac-8d28fd1c626a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:46.507193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde022d-6900-485c-aaac-38c63126abab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:46.507872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde022d-6900-485c-aaac-38c63126abab None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:46.508151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde022d-6900-485c-aaac-38c63126abab None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:46.508429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde022d-6900-485c-aaac-38c63126abab None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:46.509148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde022d-6900-485c-aaac-38c63126abab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:04:46.509458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde022d-6900-485c-aaac-38c63126abab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:46.515591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde022d-6900-485c-aaac-38c63126abab None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:46.516355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 19/38] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:46.518720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae7671-9c57-4285-8fac-8d28fd1c626a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:46.532593 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 20/39] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:04:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:46.541371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde022d-6900-485c-aaac-38c63126abab None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:46.593059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:46.667069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae7671-9c57-4285-8fac-8d28fd1c626a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:46.690413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde022d-6900-485c-aaac-38c63126abab None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:46.725751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7961982-3bcc-4f26-9264-0ee3f55fd981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:46.726837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7961982-3bcc-4f26-9264-0ee3f55fd981 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:46.727403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7961982-3bcc-4f26-9264-0ee3f55fd981 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:46.727853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7961982-3bcc-4f26-9264-0ee3f55fd981 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:46.729228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7961982-3bcc-4f26-9264-0ee3f55fd981 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:46.732221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7961982-3bcc-4f26-9264-0ee3f55fd981 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:46.742083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cff61-75d4-4c5b-9ed3-0e8c656cfa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:46.746873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cff61-75d4-4c5b-9ed3-0e8c656cfa5d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:46.746873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cff61-75d4-4c5b-9ed3-0e8c656cfa5d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:46.746873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cff61-75d4-4c5b-9ed3-0e8c656cfa5d None admin] PATH_INFO: `/v3/users/efae8da899ef465d9872282b2bb91fef` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:46.746873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cff61-75d4-4c5b-9ed3-0e8c656cfa5d None admin] RBAC: Authorizing `identity:get_user(user_id=efae8da899ef465d9872282b2bb91fef)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:46.747420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cff61-75d4-4c5b-9ed3-0e8c656cfa5d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:46.756772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 21/40] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:04:46 2022] GET /identity/v3/users/efae8da899ef465d9872282b2bb91fef => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:46.756772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cff61-75d4-4c5b-9ed3-0e8c656cfa5d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:46.876017 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:46.886225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cff61-75d4-4c5b-9ed3-0e8c656cfa5d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:46.894498 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 20/41] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:04:46 2022] POST /identity/v3/projects => generated 301 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:04:46.896444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7961982-3bcc-4f26-9264-0ee3f55fd981 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:46.934058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:46.938336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d0e9a-2a1e-44e5-8514-65161ab4886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:46.939009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d0e9a-2a1e-44e5-8514-65161ab4886e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:46.939289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d0e9a-2a1e-44e5-8514-65161ab4886e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:46.939585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d0e9a-2a1e-44e5-8514-65161ab4886e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:46.940889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d0e9a-2a1e-44e5-8514-65161ab4886e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:46.942554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d0e9a-2a1e-44e5-8514-65161ab4886e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:46.947752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 22/42] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:46.947752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d0e9a-2a1e-44e5-8514-65161ab4886e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:47.064694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7961982-3bcc-4f26-9264-0ee3f55fd981 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:47.102483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d0e9a-2a1e-44e5-8514-65161ab4886e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:47.155331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82004ed3-198c-4a41-8b8b-f3a7777583ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:47.156092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004ed3-198c-4a41-8b8b-f3a7777583ad None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:47.156377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004ed3-198c-4a41-8b8b-f3a7777583ad None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:47.156662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82004ed3-198c-4a41-8b8b-f3a7777583ad None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:47.157385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82004ed3-198c-4a41-8b8b-f3a7777583ad None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:47.158840 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82004ed3-198c-4a41-8b8b-f3a7777583ad None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:47.233333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3886ef-5561-43de-838d-160c1ba222a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:47.234045 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3886ef-5561-43de-838d-160c1ba222a9 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:47.234562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3886ef-5561-43de-838d-160c1ba222a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:47.234680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3886ef-5561-43de-838d-160c1ba222a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:47.236121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3886ef-5561-43de-838d-160c1ba222a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:47.238366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3886ef-5561-43de-838d-160c1ba222a9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:47.241219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 21/43] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:47.242593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3886ef-5561-43de-838d-160c1ba222a9 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:47.329509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 23/44] 10.222.0.64 () {60 vars in 1131 bytes} [Fri Sep 23 01:04:46 2022] POST /identity/v3/roles => generated 223 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:04:47.330305 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82004ed3-198c-4a41-8b8b-f3a7777583ad None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:47.374698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:47.378577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:47.404490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3886ef-5561-43de-838d-160c1ba222a9 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:47.467219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2b9f6-5d49-4ac5-9067-f8a652e98611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:47.468140 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b9f6-5d49-4ac5-9067-f8a652e98611 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:47.468648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b9f6-5d49-4ac5-9067-f8a652e98611 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:47.469138 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b9f6-5d49-4ac5-9067-f8a652e98611 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:47.470092 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2b9f6-5d49-4ac5-9067-f8a652e98611 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:47.472118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2b9f6-5d49-4ac5-9067-f8a652e98611 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:47.498860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82004ed3-198c-4a41-8b8b-f3a7777583ad None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:47.548210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea32fd-a120-448f-ace9-d6d76dfb58bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:47.549221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea32fd-a120-448f-ace9-d6d76dfb58bb None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:47.549606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea32fd-a120-448f-ace9-d6d76dfb58bb None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:47.550035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea32fd-a120-448f-ace9-d6d76dfb58bb None admin] PATH_INFO: `/v3/domains/default/users/efae8da899ef465d9872282b2bb91fef/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:47.553090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea32fd-a120-448f-ace9-d6d76dfb58bb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=efae8da899ef465d9872282b2bb91fef, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:47.554871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea32fd-a120-448f-ace9-d6d76dfb58bb None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:47.630571 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 22/45] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:04:47 2022] POST /identity/v3/projects => generated 312 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:04:47.796458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 24/46] 10.222.0.64 () {58 vars in 1361 bytes} [Fri Sep 23 01:04:47 2022] PUT /identity/v3/domains/default/users/efae8da899ef465d9872282b2bb91fef/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:04:49.851257 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c665d23-346e-403a-a0f3-06563d5bf800 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:49.851881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c665d23-346e-403a-a0f3-06563d5bf800 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:49.851881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c665d23-346e-403a-a0f3-06563d5bf800 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:49.853314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 23/47] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:49.859502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db36299-178e-4a17-acf2-aad0fe12cb67 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:49.860084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db36299-178e-4a17-acf2-aad0fe12cb67 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:49.860239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db36299-178e-4a17-acf2-aad0fe12cb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:49.879769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-4db36299-178e-4a17-acf2-aad0fe12cb67 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:49.881548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4db36299-178e-4a17-acf2-aad0fe12cb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:49.960041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 25/48] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:49.966617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88f877-5cb3-4d44-87ff-7444977edbf7 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:49.967067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88f877-5cb3-4d44-87ff-7444977edbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:49.967349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88f877-5cb3-4d44-87ff-7444977edbf7 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:49.968776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 24/49] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:49.974959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4233da5-4fd1-4691-8e3f-529f537653e1 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:49.975258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4233da5-4fd1-4691-8e3f-529f537653e1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:49.975589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4233da5-4fd1-4691-8e3f-529f537653e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:49.994395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b4233da5-4fd1-4691-8e3f-529f537653e1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:49.995653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b4233da5-4fd1-4691-8e3f-529f537653e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:50.006538 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:50.071674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 26/50] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:50.232806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88f877-5cb3-4d44-87ff-7444977edbf7 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:50.343215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4233da5-4fd1-4691-8e3f-529f537653e1 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:50.348887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88f877-5cb3-4d44-87ff-7444977edbf7 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:50.395946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6666952-5ab8-4788-a532-0412935bd045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:50.396705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6666952-5ab8-4788-a532-0412935bd045 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:50.397105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6666952-5ab8-4788-a532-0412935bd045 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:50.397394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6666952-5ab8-4788-a532-0412935bd045 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:50.398826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6666952-5ab8-4788-a532-0412935bd045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:50.400399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6666952-5ab8-4788-a532-0412935bd045 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:50.402246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 25/51] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:04:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:50.408264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6666952-5ab8-4788-a532-0412935bd045 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:50.418048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:50.462158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4233da5-4fd1-4691-8e3f-529f537653e1 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:50.504376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a79c81-8bc3-400e-9c28-e19f8c21813c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:50.504829 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a79c81-8bc3-400e-9c28-e19f8c21813c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:50.505724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a79c81-8bc3-400e-9c28-e19f8c21813c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:50.506000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a79c81-8bc3-400e-9c28-e19f8c21813c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:50.517622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a79c81-8bc3-400e-9c28-e19f8c21813c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-79a79c81-8bc3-400e-9c28-e19f8c21813c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:04:50.522746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:04:50.523904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:04:50.524693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 27/52] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:04:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:04:50.527997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6666952-5ab8-4788-a532-0412935bd045 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:50.530294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a79c81-8bc3-400e-9c28-e19f8c21813c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:50.535767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:50.568976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f67fa4-1b8a-48bf-960f-4de50643fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:50.569694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f67fa4-1b8a-48bf-960f-4de50643fee8 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:50.569980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f67fa4-1b8a-48bf-960f-4de50643fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:50.570262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f67fa4-1b8a-48bf-960f-4de50643fee8 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:50.570947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f67fa4-1b8a-48bf-960f-4de50643fee8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:50.572489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f67fa4-1b8a-48bf-960f-4de50643fee8 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:50.644643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a79c81-8bc3-400e-9c28-e19f8c21813c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:50.736219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 26/53] 10.222.0.64 () {60 vars in 1140 bytes} [Fri Sep 23 01:04:50 2022] POST /identity/v3/projects => generated 298 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:04:50.802556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d1cf6-abc2-4b31-a027-aa0eb5b32c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:50.802556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d1cf6-abc2-4b31-a027-aa0eb5b32c43 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:50.802556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d1cf6-abc2-4b31-a027-aa0eb5b32c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:50.802556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d1cf6-abc2-4b31-a027-aa0eb5b32c43 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:50.803067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d1cf6-abc2-4b31-a027-aa0eb5b32c43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:04:50.803294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d1cf6-abc2-4b31-a027-aa0eb5b32c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:50.804821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d1cf6-abc2-4b31-a027-aa0eb5b32c43 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:50.816061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 28/54] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:04:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:04:50.823836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f67fa4-1b8a-48bf-960f-4de50643fee8 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:50.905346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:50.967073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f67fa4-1b8a-48bf-960f-4de50643fee8 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:51.012363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaa0f9-6ffd-4608-b573-fcbe5b2bf3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:51.013053 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa0f9-6ffd-4608-b573-fcbe5b2bf3eb None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:51.013356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa0f9-6ffd-4608-b573-fcbe5b2bf3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:51.013701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa0f9-6ffd-4608-b573-fcbe5b2bf3eb None admin] PATH_INFO: `/v3/users/efae8da899ef465d9872282b2bb91fef` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:51.015060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edaa0f9-6ffd-4608-b573-fcbe5b2bf3eb None admin] RBAC: Authorizing `identity:get_user(user_id=efae8da899ef465d9872282b2bb91fef)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:51.016667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edaa0f9-6ffd-4608-b573-fcbe5b2bf3eb None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:51.020387 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 27/55] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:04:50 2022] GET /identity/v3/users/efae8da899ef465d9872282b2bb91fef => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:51.025973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d1cf6-abc2-4b31-a027-aa0eb5b32c43 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:51.117724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:51.157048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d1cf6-abc2-4b31-a027-aa0eb5b32c43 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:51.199212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e3abc-d644-4c3c-8e22-385adfc8cf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:51.199916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e3abc-d644-4c3c-8e22-385adfc8cf30 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:51.200269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e3abc-d644-4c3c-8e22-385adfc8cf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:51.200550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e3abc-d644-4c3c-8e22-385adfc8cf30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:51.201804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e3abc-d644-4c3c-8e22-385adfc8cf30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:51.203279 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e3abc-d644-4c3c-8e22-385adfc8cf30 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:51.205197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 29/56] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:04:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:51.211583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaa0f9-6ffd-4608-b573-fcbe5b2bf3eb None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:51.341161 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaa0f9-6ffd-4608-b573-fcbe5b2bf3eb None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:51.376040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:51.388591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8811ca-cd04-4bde-bfa1-84dd5f3224e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:51.389200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8811ca-cd04-4bde-bfa1-84dd5f3224e3 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:51.389484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8811ca-cd04-4bde-bfa1-84dd5f3224e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:51.389786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8811ca-cd04-4bde-bfa1-84dd5f3224e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:51.390543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8811ca-cd04-4bde-bfa1-84dd5f3224e3 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.domain.id=default, user.id=efae8da899ef465d9872282b2bb91fef) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:04:51.390853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8811ca-cd04-4bde-bfa1-84dd5f3224e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:51.392371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8811ca-cd04-4bde-bfa1-84dd5f3224e3 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:51.400999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 28/57] 10.222.0.64 () {56 vars in 1322 bytes} [Fri Sep 23 01:04:51 2022] GET /identity/v3/role_assignments?user.id=efae8da899ef465d9872282b2bb91fef&scope.domain.id=default&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 530 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:04:53.111261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5408677-7421-49a3-9e1c-7565ca3be359 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:53.111911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5408677-7421-49a3-9e1c-7565ca3be359 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:53.111911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5408677-7421-49a3-9e1c-7565ca3be359 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:53.113157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 30/58] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:53.119701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4116-98fe-4753-a3ee-8d8b018b799e None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:53.120231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4116-98fe-4753-a3ee-8d8b018b799e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:53.120335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a4116-98fe-4753-a3ee-8d8b018b799e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:53.140902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-2e1a4116-98fe-4753-a3ee-8d8b018b799e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:53.142537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2e1a4116-98fe-4753-a3ee-8d8b018b799e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:53.236774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 29/59] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:04:53.525623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5408677-7421-49a3-9e1c-7565ca3be359 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:53.654731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5408677-7421-49a3-9e1c-7565ca3be359 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:53.704180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f813371-f871-40b0-8f49-fbf9aa656034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:53.705104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f813371-f871-40b0-8f49-fbf9aa656034 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:53.705436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f813371-f871-40b0-8f49-fbf9aa656034 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:53.705762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f813371-f871-40b0-8f49-fbf9aa656034 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:53.707053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f813371-f871-40b0-8f49-fbf9aa656034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:53.708691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f813371-f871-40b0-8f49-fbf9aa656034 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:53.710496 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 31/60] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:04:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:53.717494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a4116-98fe-4753-a3ee-8d8b018b799e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:53.854419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a4116-98fe-4753-a3ee-8d8b018b799e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:53.888483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:53.916082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a17b8a-0088-41f5-b03d-a1a7eb73fa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:53.917189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a17b8a-0088-41f5-b03d-a1a7eb73fa02 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:53.917702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a17b8a-0088-41f5-b03d-a1a7eb73fa02 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:53.918215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a17b8a-0088-41f5-b03d-a1a7eb73fa02 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:53.919494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a17b8a-0088-41f5-b03d-a1a7eb73fa02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:53.922536 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a17b8a-0088-41f5-b03d-a1a7eb73fa02 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:54.064395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 30/61] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:04:53 2022] POST /identity/v3/users => generated 275 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:04:58.032962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f5f28-765b-4e5c-b726-2c36d2bab0d2 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:58.034461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f5f28-765b-4e5c-b726-2c36d2bab0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:58.034987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f5f28-765b-4e5c-b726-2c36d2bab0d2 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:58.037071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 32/62] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:58.054345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ed42-50d4-4549-ac0b-1bc97d2404e3 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:58.055900 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ed42-50d4-4549-ac0b-1bc97d2404e3 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:58.056663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ed42-50d4-4549-ac0b-1bc97d2404e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:58.098886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-6e85ed42-50d4-4549-ac0b-1bc97d2404e3 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:58.100824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6e85ed42-50d4-4549-ac0b-1bc97d2404e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:58.223048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 31/63] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:58.480351 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c2e14-8411-4578-b864-f778f13afa89 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:58.481130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c2e14-8411-4578-b864-f778f13afa89 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:58.481657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c2e14-8411-4578-b864-f778f13afa89 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:58.484085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 33/64] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:58.490960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184c95f-02a4-4d87-abb7-e81a83670679 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:58.491703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184c95f-02a4-4d87-abb7-e81a83670679 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:58.492249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184c95f-02a4-4d87-abb7-e81a83670679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:58.532201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d184c95f-02a4-4d87-abb7-e81a83670679 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:58.534975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d184c95f-02a4-4d87-abb7-e81a83670679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:58.695391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c2e14-8411-4578-b864-f778f13afa89 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:58.699522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:58.722771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 32/65] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:58.899029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a8e58-3c01-4816-b9ad-18532c82f7f1 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:58.899525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a8e58-3c01-4816-b9ad-18532c82f7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:58.904294 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a8e58-3c01-4816-b9ad-18532c82f7f1 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:58.906527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 33/66] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:58.917268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5d3ea-3a25-4329-b575-21dfdd5208a9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:58.917562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5d3ea-3a25-4329-b575-21dfdd5208a9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:58.917924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5d3ea-3a25-4329-b575-21dfdd5208a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:58.955544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-70e5d3ea-3a25-4329-b575-21dfdd5208a9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:58.956954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-70e5d3ea-3a25-4329-b575-21dfdd5208a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:58.961065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c2e14-8411-4578-b864-f778f13afa89 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:59.032274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:59.037841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee548a7f-499a-4202-b788-d493ea0ee41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:59.038592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee548a7f-499a-4202-b788-d493ea0ee41c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:59.038877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee548a7f-499a-4202-b788-d493ea0ee41c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:59.039154 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee548a7f-499a-4202-b788-d493ea0ee41c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:59.062189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee548a7f-499a-4202-b788-d493ea0ee41c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-ee548a7f-499a-4202-b788-d493ea0ee41c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:04:59.067856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:04:59.069021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:04:59.069848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 34/67] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:04:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:04:59.075162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca24ed-2f90-4a9a-a4be-71a94832ebb8 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:59.075162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca24ed-2f90-4a9a-a4be-71a94832ebb8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:59.075162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca24ed-2f90-4a9a-a4be-71a94832ebb8 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:59.075162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 35/68] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:04:59.078157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca24ed-2f90-4a9a-a4be-71a94832ebb8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:59.095363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 34/69] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:59.105164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5d3ea-3a25-4329-b575-21dfdd5208a9 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:59.191607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:59.241767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca24ed-2f90-4a9a-a4be-71a94832ebb8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:59.291635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c79335-2689-4252-a0e4-720a54f48466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:59.293052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c79335-2689-4252-a0e4-720a54f48466 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:59.294178 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c79335-2689-4252-a0e4-720a54f48466 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:59.295008 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c79335-2689-4252-a0e4-720a54f48466 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:59.301202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5d3ea-3a25-4329-b575-21dfdd5208a9 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:59.316052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c79335-2689-4252-a0e4-720a54f48466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-e9c79335-2689-4252-a0e4-720a54f48466 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:04:59.330950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:04:59.332724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 36/70] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:04:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:04:59.340690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2910b-6223-4f1c-bea7-fa23bdbeddb5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:59.341415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2910b-6223-4f1c-bea7-fa23bdbeddb5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:59.341415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2910b-6223-4f1c-bea7-fa23bdbeddb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:59.365779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedc3aa-f43e-47d0-85a6-682d1b553c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:59.366812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedc3aa-f43e-47d0-85a6-682d1b553c79 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:59.367137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedc3aa-f43e-47d0-85a6-682d1b553c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:59.367461 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedc3aa-f43e-47d0-85a6-682d1b553c79 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:59.368334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fedc3aa-f43e-47d0-85a6-682d1b553c79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:04:59.368649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fedc3aa-f43e-47d0-85a6-682d1b553c79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:59.370437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fedc3aa-f43e-47d0-85a6-682d1b553c79 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:59.370854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-0fc2910b-6223-4f1c-bea7-fa23bdbeddb5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:04:59.372800 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0fc2910b-6223-4f1c-bea7-fa23bdbeddb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:04:59.385519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 35/71] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:04:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:59.386565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedc3aa-f43e-47d0-85a6-682d1b553c79 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:59.427420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:59.492564 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 37/72] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:04:59.492564 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2910b-6223-4f1c-bea7-fa23bdbeddb5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:59.528663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedc3aa-f43e-47d0-85a6-682d1b553c79 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:59.578433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:59.579762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32420c-9ddc-4a38-9057-ae0784c301bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:59.580919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32420c-9ddc-4a38-9057-ae0784c301bf None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:59.581531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32420c-9ddc-4a38-9057-ae0784c301bf None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:59.582130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32420c-9ddc-4a38-9057-ae0784c301bf None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:59.583387 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32420c-9ddc-4a38-9057-ae0784c301bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:04:59.584572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32420c-9ddc-4a38-9057-ae0784c301bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:59.587037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32420c-9ddc-4a38-9057-ae0784c301bf None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:59.600459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 36/73] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:04:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:59.602133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32420c-9ddc-4a38-9057-ae0784c301bf None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:59.642909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2910b-6223-4f1c-bea7-fa23bdbeddb5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:59.692711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40f261-4b4a-4e0c-b403-bc7dbf45a376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:59.693549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f261-4b4a-4e0c-b403-bc7dbf45a376 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:59.694348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f261-4b4a-4e0c-b403-bc7dbf45a376 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:59.694899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40f261-4b4a-4e0c-b403-bc7dbf45a376 None admin] PATH_INFO: `/v3/users/efae8da899ef465d9872282b2bb91fef` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:59.698234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f261-4b4a-4e0c-b403-bc7dbf45a376 None admin] RBAC: Authorizing `identity:get_user(user_id=efae8da899ef465d9872282b2bb91fef)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:59.700401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40f261-4b4a-4e0c-b403-bc7dbf45a376 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:59.702557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 38/74] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:04:59 2022] GET /identity/v3/users/efae8da899ef465d9872282b2bb91fef => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:59.709789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40f261-4b4a-4e0c-b403-bc7dbf45a376 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:59.744082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:59.756681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32420c-9ddc-4a38-9057-ae0784c301bf None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:59.799441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ad056-0fb6-4a4d-8448-b57e105473c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:59.800754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ad056-0fb6-4a4d-8448-b57e105473c0 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:59.801655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ad056-0fb6-4a4d-8448-b57e105473c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:59.802459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ad056-0fb6-4a4d-8448-b57e105473c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:59.827830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ad056-0fb6-4a4d-8448-b57e105473c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-940ad056-0fb6-4a4d-8448-b57e105473c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:04:59.836142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:04:59.837073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:04:59.837840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:04:59.838496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 37/75] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:04:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:04:59.839311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ad056-0fb6-4a4d-8448-b57e105473c0 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:59.867398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40f261-4b4a-4e0c-b403-bc7dbf45a376 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:04:59.897643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:04:59.918350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4873e-3436-45f3-a439-b54008baec8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:04:59.919455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4873e-3436-45f3-a439-b54008baec8b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:04:59.919988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4873e-3436-45f3-a439-b54008baec8b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:04:59.920480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4873e-3436-45f3-a439-b54008baec8b None admin] PATH_INFO: `/v3/users/37677eab531b4f658648830316674597` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:04:59.950669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4873e-3436-45f3-a439-b54008baec8b None admin] RBAC: Authorizing `identity:get_user(user_id=37677eab531b4f658648830316674597)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:04:59.953580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4873e-3436-45f3-a439-b54008baec8b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:04:59.957304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 39/76] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:04:59 2022] GET /identity/v3/users/37677eab531b4f658648830316674597 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:04:59.959160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4873e-3436-45f3-a439-b54008baec8b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:04:59.993118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ad056-0fb6-4a4d-8448-b57e105473c0 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:00.041500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a5435-6dd0-43af-b5b9-563998b3263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:00.042122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5435-6dd0-43af-b5b9-563998b3263c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:00.042405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5435-6dd0-43af-b5b9-563998b3263c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:00.042681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5435-6dd0-43af-b5b9-563998b3263c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:00.065440 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:00.097628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4873e-3436-45f3-a439-b54008baec8b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:00.145820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a5435-6dd0-43af-b5b9-563998b3263c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:00.147929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a5435-6dd0-43af-b5b9-563998b3263c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:00.151506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5653fe-937a-42d1-b1e0-27933b81a8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:00.152227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5653fe-937a-42d1-b1e0-27933b81a8cd None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:00.152514 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5653fe-937a-42d1-b1e0-27933b81a8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:00.152877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5653fe-937a-42d1-b1e0-27933b81a8cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:00.164932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5653fe-937a-42d1-b1e0-27933b81a8cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-af5653fe-937a-42d1-b1e0-27933b81a8cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:00.169984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:00.171105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 40/77] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:04:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:00.172332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5653fe-937a-42d1-b1e0-27933b81a8cd None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:00.217378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:00.298830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5653fe-937a-42d1-b1e0-27933b81a8cd None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:00.309538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 38/78] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:04:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:00.311994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0a816-b34f-4c21-83da-b0c4215d583e None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:00.312509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0a816-b34f-4c21-83da-b0c4215d583e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:00.313027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0a816-b34f-4c21-83da-b0c4215d583e None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:00.315805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 39/79] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:00.317349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0a816-b34f-4c21-83da-b0c4215d583e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:00.360097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10b44d-50e9-4933-b4fb-9f0730acb685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:00.361075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10b44d-50e9-4933-b4fb-9f0730acb685 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:00.361575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10b44d-50e9-4933-b4fb-9f0730acb685 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:00.362058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10b44d-50e9-4933-b4fb-9f0730acb685 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:00.363292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10b44d-50e9-4933-b4fb-9f0730acb685 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:00.363862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10b44d-50e9-4933-b4fb-9f0730acb685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:00.366987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10b44d-50e9-4933-b4fb-9f0730acb685 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:00.381517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 41/80] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:05:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:00.382469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10b44d-50e9-4933-b4fb-9f0730acb685 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:00.393899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:00.467031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0a816-b34f-4c21-83da-b0c4215d583e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:00.517079 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428886-623c-47cf-b9a0-784d3812cc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:00.518016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428886-623c-47cf-b9a0-784d3812cc4f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:00.518476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428886-623c-47cf-b9a0-784d3812cc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:00.518864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:00.518978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c428886-623c-47cf-b9a0-784d3812cc4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:00.523360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10b44d-50e9-4933-b4fb-9f0730acb685 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:00.566330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e4ddc-1969-4832-8a7f-525601102cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:00.566944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e4ddc-1969-4832-8a7f-525601102cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:00.567268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e4ddc-1969-4832-8a7f-525601102cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:00.567544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e4ddc-1969-4832-8a7f-525601102cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:00.568267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e4ddc-1969-4832-8a7f-525601102cc0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:00.568577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e4ddc-1969-4832-8a7f-525601102cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:00.570298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e4ddc-1969-4832-8a7f-525601102cc0 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:00.579869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 42/81] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:00.581971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e4ddc-1969-4832-8a7f-525601102cc0 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:00.630219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c428886-623c-47cf-b9a0-784d3812cc4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:00.632968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c428886-623c-47cf-b9a0-784d3812cc4f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:00.723028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e4ddc-1969-4832-8a7f-525601102cc0 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:00.731855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:00.776729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a2d9d-ba17-4bfc-9fcc-aea3c27bcfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:00.777852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a2d9d-ba17-4bfc-9fcc-aea3c27bcfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:00.778307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a2d9d-ba17-4bfc-9fcc-aea3c27bcfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:00.778752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a2d9d-ba17-4bfc-9fcc-aea3c27bcfb8 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:00.781410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a2d9d-ba17-4bfc-9fcc-aea3c27bcfb8 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:00.784312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a2d9d-ba17-4bfc-9fcc-aea3c27bcfb8 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:00.788522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 43/82] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:00 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:00.796377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 40/83] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:00.796377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428886-623c-47cf-b9a0-784d3812cc4f None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:00.796772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa2cd0-e70a-4eb0-ba09-d23f44333e5f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:00.796772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa2cd0-e70a-4eb0-ba09-d23f44333e5f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:00.796772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa2cd0-e70a-4eb0-ba09-d23f44333e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:00.823715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c7aa2cd0-e70a-4eb0-ba09-d23f44333e5f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:00.825862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c7aa2cd0-e70a-4eb0-ba09-d23f44333e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:00.867520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:00.920575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 44/84] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:00.921641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa2cd0-e70a-4eb0-ba09-d23f44333e5f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:00.950039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c428886-623c-47cf-b9a0-784d3812cc4f None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:01.000762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ab0b-0aa9-4813-b762-30c18d6bb339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:01.001386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ab0b-0aa9-4813-b762-30c18d6bb339 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:01.001667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ab0b-0aa9-4813-b762-30c18d6bb339 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:01.001986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ab0b-0aa9-4813-b762-30c18d6bb339 None admin] PATH_INFO: `/v3/users/37677eab531b4f658648830316674597` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:01.003242 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83ab0b-0aa9-4813-b762-30c18d6bb339 None admin] RBAC: Authorizing `identity:get_user(user_id=37677eab531b4f658648830316674597)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:01.005434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83ab0b-0aa9-4813-b762-30c18d6bb339 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:01.012282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 41/85] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:00 2022] GET /identity/v3/users/37677eab531b4f658648830316674597 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:01.012282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ab0b-0aa9-4813-b762-30c18d6bb339 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:01.032124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:01.116979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa2cd0-e70a-4eb0-ba09-d23f44333e5f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:01.128824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ab0b-0aa9-4813-b762-30c18d6bb339 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:01.169937 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5b895-fa39-453e-a3df-70535c4354a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:01.170901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5b895-fa39-453e-a3df-70535c4354a6 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:01.170901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5b895-fa39-453e-a3df-70535c4354a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:01.171156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5b895-fa39-453e-a3df-70535c4354a6 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:01.172649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5b895-fa39-453e-a3df-70535c4354a6 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:01.174817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5b895-fa39-453e-a3df-70535c4354a6 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:01.177323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 42/86] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:01 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:01.178230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5b895-fa39-453e-a3df-70535c4354a6 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:01.190582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5edaf-10f4-4a2b-9be9-6af666768e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:01.191748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5edaf-10f4-4a2b-9be9-6af666768e72 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:01.192292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5edaf-10f4-4a2b-9be9-6af666768e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:01.192784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5edaf-10f4-4a2b-9be9-6af666768e72 None admin] PATH_INFO: `/v3/users/37677eab531b4f658648830316674597` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:01.195114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5edaf-10f4-4a2b-9be9-6af666768e72 None admin] RBAC: Authorizing `identity:get_user(user_id=37677eab531b4f658648830316674597)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:01.197983 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5edaf-10f4-4a2b-9be9-6af666768e72 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:01.201226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 45/87] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:00 2022] GET /identity/v3/users/37677eab531b4f658648830316674597 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:01.202666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5edaf-10f4-4a2b-9be9-6af666768e72 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:01.248262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:01.308948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:01.331493 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5b895-fa39-453e-a3df-70535c4354a6 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:01.357399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5edaf-10f4-4a2b-9be9-6af666768e72 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:01.371706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3cbfe-11ac-458c-9dfa-7afd46d03515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:01.372332 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3cbfe-11ac-458c-9dfa-7afd46d03515 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:01.372702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3cbfe-11ac-458c-9dfa-7afd46d03515 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:01.372951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3cbfe-11ac-458c-9dfa-7afd46d03515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:01.373794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3cbfe-11ac-458c-9dfa-7afd46d03515 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005, user.id=efae8da899ef465d9872282b2bb91fef) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:01.374108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3cbfe-11ac-458c-9dfa-7afd46d03515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:01.375603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b3cbfe-11ac-458c-9dfa-7afd46d03515 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:01.381341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 43/88] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:01 2022] GET /identity/v3/role_assignments?user.id=efae8da899ef465d9872282b2bb91fef&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:01.382166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3cbfe-11ac-458c-9dfa-7afd46d03515 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:01.402759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d535fad-b420-4e58-ad57-4e134f86b262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:01.403460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d535fad-b420-4e58-ad57-4e134f86b262 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:01.403778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d535fad-b420-4e58-ad57-4e134f86b262 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:01.404227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d535fad-b420-4e58-ad57-4e134f86b262 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:01.509506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d535fad-b420-4e58-ad57-4e134f86b262 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:01.511711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d535fad-b420-4e58-ad57-4e134f86b262 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:01.558361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:01.579151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b3cbfe-11ac-458c-9dfa-7afd46d03515 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:01.631405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155066a-1d8f-415d-8670-2647dbe61fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:01.632740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155066a-1d8f-415d-8670-2647dbe61fae None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:01.633386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155066a-1d8f-415d-8670-2647dbe61fae None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:01.633881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155066a-1d8f-415d-8670-2647dbe61fae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:01.635348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155066a-1d8f-415d-8670-2647dbe61fae None admin] RBAC: Adding query filter params (role.id=e5a65ae2d0884e419cdbc70c9223eac2, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005, user.id=37677eab531b4f658648830316674597) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:01.635922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155066a-1d8f-415d-8670-2647dbe61fae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:01.638885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155066a-1d8f-415d-8670-2647dbe61fae None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:01.659586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 44/89] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:01 2022] GET /identity/v3/role_assignments?user.id=37677eab531b4f658648830316674597&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:01.659586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155066a-1d8f-415d-8670-2647dbe61fae None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:01.674869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 46/90] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:01.676277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d535fad-b420-4e58-ad57-4e134f86b262 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:01.694514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:01.878413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155066a-1d8f-415d-8670-2647dbe61fae None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:01.907708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d535fad-b420-4e58-ad57-4e134f86b262 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:01.935725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b38896-0c54-4826-8f5c-14f1f14ae863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:01.936742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b38896-0c54-4826-8f5c-14f1f14ae863 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:01.937228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b38896-0c54-4826-8f5c-14f1f14ae863 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:01.937728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b38896-0c54-4826-8f5c-14f1f14ae863 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:01.956665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:01.976335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335deaf3-a74d-4c4f-9a9e-dfbf63ca858d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:01.977782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335deaf3-a74d-4c4f-9a9e-dfbf63ca858d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:01.978272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335deaf3-a74d-4c4f-9a9e-dfbf63ca858d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:01.979001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335deaf3-a74d-4c4f-9a9e-dfbf63ca858d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:01.982214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335deaf3-a74d-4c4f-9a9e-dfbf63ca858d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:01.986173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335deaf3-a74d-4c4f-9a9e-dfbf63ca858d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:01.991717 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 47/91] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:05:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:01.993495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335deaf3-a74d-4c4f-9a9e-dfbf63ca858d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:02.043818 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b38896-0c54-4826-8f5c-14f1f14ae863 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:02.045879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b38896-0c54-4826-8f5c-14f1f14ae863 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:02.065953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:02.165073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335deaf3-a74d-4c4f-9a9e-dfbf63ca858d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:02.227981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922614-29ba-4a77-b7a7-e037afdf2285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:02.228668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922614-29ba-4a77-b7a7-e037afdf2285 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:02.228999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922614-29ba-4a77-b7a7-e037afdf2285 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:02.229281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29922614-29ba-4a77-b7a7-e037afdf2285 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:02.230614 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29922614-29ba-4a77-b7a7-e037afdf2285 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:02.232235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29922614-29ba-4a77-b7a7-e037afdf2285 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:02.234496 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 48/92] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:01 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:02.235272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922614-29ba-4a77-b7a7-e037afdf2285 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:02.258856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 45/93] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:02.262946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b38896-0c54-4826-8f5c-14f1f14ae863 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:02.360077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:02.449759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29922614-29ba-4a77-b7a7-e037afdf2285 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:02.482808 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b38896-0c54-4826-8f5c-14f1f14ae863 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:02.509881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:02.514844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc97d71-c3c0-45d1-a7f9-81cbe98c65c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:02.515938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc97d71-c3c0-45d1-a7f9-81cbe98c65c0 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:02.516791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc97d71-c3c0-45d1-a7f9-81cbe98c65c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:02.517251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc97d71-c3c0-45d1-a7f9-81cbe98c65c0 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:02.518246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc97d71-c3c0-45d1-a7f9-81cbe98c65c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:02.520118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc97d71-c3c0-45d1-a7f9-81cbe98c65c0 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:02.549622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0118382-24f1-45e2-81a3-72ed980874cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:02.550791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0118382-24f1-45e2-81a3-72ed980874cb None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:02.551350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0118382-24f1-45e2-81a3-72ed980874cb None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:02.551986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0118382-24f1-45e2-81a3-72ed980874cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:02.553416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0118382-24f1-45e2-81a3-72ed980874cb None admin] RBAC: Adding query filter params (role.id=b9c95d84eaca4444b2c7b8b5eed61a59, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005, user.id=37677eab531b4f658648830316674597) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:02.554019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0118382-24f1-45e2-81a3-72ed980874cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:02.556893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0118382-24f1-45e2-81a3-72ed980874cb None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:02.574799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 46/94] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:02 2022] GET /identity/v3/role_assignments?user.id=37677eab531b4f658648830316674597&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=b9c95d84eaca4444b2c7b8b5eed61a59 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:02.574799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0118382-24f1-45e2-81a3-72ed980874cb None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:02.657116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 49/95] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:05:02 2022] POST /identity/v3/users => generated 289 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:05:02.815838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0118382-24f1-45e2-81a3-72ed980874cb None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:02.907544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e9e88-5526-4a0a-9d3f-ce263ee55a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:02.908697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e9e88-5526-4a0a-9d3f-ce263ee55a3f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:02.909200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e9e88-5526-4a0a-9d3f-ce263ee55a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:02.909700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e9e88-5526-4a0a-9d3f-ce263ee55a3f None admin] PATH_INFO: `/v3/projects/f2b4f9336b334662b06b49fda57b2fd1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:02.916475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e9e88-5526-4a0a-9d3f-ce263ee55a3f None admin] RBAC: Authorizing `identity:get_project(project_id=f2b4f9336b334662b06b49fda57b2fd1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:02.919585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e9e88-5526-4a0a-9d3f-ce263ee55a3f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:02.923307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 47/96] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:02 2022] GET /identity/v3/projects/f2b4f9336b334662b06b49fda57b2fd1 => generated 312 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:02.934239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc97d71-c3c0-45d1-a7f9-81cbe98c65c0 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:03.170771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc97d71-c3c0-45d1-a7f9-81cbe98c65c0 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:03.214962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:03.248032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909008e-c9a0-4208-8779-01f8d84c5746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:03.249124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909008e-c9a0-4208-8779-01f8d84c5746 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:03.249655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909008e-c9a0-4208-8779-01f8d84c5746 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:03.250145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909008e-c9a0-4208-8779-01f8d84c5746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:03.251580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909008e-c9a0-4208-8779-01f8d84c5746 None admin] RBAC: Adding query filter params (role.id=e5a65ae2d0884e419cdbc70c9223eac2, scope.project.id=f2b4f9336b334662b06b49fda57b2fd1, user.id=37677eab531b4f658648830316674597) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:03.252193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909008e-c9a0-4208-8779-01f8d84c5746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:03.255525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909008e-c9a0-4208-8779-01f8d84c5746 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:03.279482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 50/97] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:02 2022] GET /identity/v3/role_assignments?user.id=37677eab531b4f658648830316674597&scope.project.id=f2b4f9336b334662b06b49fda57b2fd1&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 259 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:04.683347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37294dbe-867b-439a-9365-722d070e3a9e None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:04.684026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37294dbe-867b-439a-9365-722d070e3a9e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:04.684026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37294dbe-867b-439a-9365-722d070e3a9e None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:04.685334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 48/98] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:04.695598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1307aa-4b2c-4aba-a9d6-569b75232adb None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:04.696280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1307aa-4b2c-4aba-a9d6-569b75232adb None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:04.696280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1307aa-4b2c-4aba-a9d6-569b75232adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:04.721075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5a1307aa-4b2c-4aba-a9d6-569b75232adb None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:04.722324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5a1307aa-4b2c-4aba-a9d6-569b75232adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:04.814439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 51/99] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:05.762449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b63dc-fcd8-4787-9566-9b97e035ae03 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:05.762992 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b63dc-fcd8-4787-9566-9b97e035ae03 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:05.763458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b63dc-fcd8-4787-9566-9b97e035ae03 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:05.766384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 49/100] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:05.779116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2a4c7-b142-4a1d-b69a-1a098e3436f9 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:05.779657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2a4c7-b142-4a1d-b69a-1a098e3436f9 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:05.780153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2a4c7-b142-4a1d-b69a-1a098e3436f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:05.820440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-58f2a4c7-b142-4a1d-b69a-1a098e3436f9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:05.822548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-58f2a4c7-b142-4a1d-b69a-1a098e3436f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:05.868575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b63dc-fcd8-4787-9566-9b97e035ae03 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:06.037697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 52/101] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:06.052646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:06.146758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b63dc-fcd8-4787-9566-9b97e035ae03 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:06.210667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6075a6e-63f6-44f8-9b42-62c3dce3281c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:06.211435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6075a6e-63f6-44f8-9b42-62c3dce3281c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:06.211975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6075a6e-63f6-44f8-9b42-62c3dce3281c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:06.212475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6075a6e-63f6-44f8-9b42-62c3dce3281c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:06.226859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6075a6e-63f6-44f8-9b42-62c3dce3281c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:06.231032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a1a0d-2b2f-48a9-9359-50075b69568b None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:06.232013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a1a0d-2b2f-48a9-9359-50075b69568b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:06.232785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a1a0d-2b2f-48a9-9359-50075b69568b None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-c6075a6e-63f6-44f8-9b42-62c3dce3281c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:06.235074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:06.241000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:06.242456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 50/102] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:05:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:06.245332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 53/103] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:06 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:05:06.249593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6075a6e-63f6-44f8-9b42-62c3dce3281c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:06.253967 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683d2ac-aba8-4860-af67-b77899b685e0 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:06.254453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683d2ac-aba8-4860-af67-b77899b685e0 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:06.255368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683d2ac-aba8-4860-af67-b77899b685e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:06.306666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-1683d2ac-aba8-4860-af67-b77899b685e0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:06.308336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1683d2ac-aba8-4860-af67-b77899b685e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:06.407439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:06.469167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 54/104] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:06.498495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6075a6e-63f6-44f8-9b42-62c3dce3281c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:06.508708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:06.549541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4fe38-bc9b-4e9b-9469-ec7c8fcc9ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:06.550182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4fe38-bc9b-4e9b-9469-ec7c8fcc9ffb None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:06.550485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4fe38-bc9b-4e9b-9469-ec7c8fcc9ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:06.550743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4fe38-bc9b-4e9b-9469-ec7c8fcc9ffb None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:06.551528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4fe38-bc9b-4e9b-9469-ec7c8fcc9ffb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:06.551869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4fe38-bc9b-4e9b-9469-ec7c8fcc9ffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:06.553479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4fe38-bc9b-4e9b-9469-ec7c8fcc9ffb None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:06.570618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 51/105] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:06.576502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683d2ac-aba8-4860-af67-b77899b685e0 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:06.755722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683d2ac-aba8-4860-af67-b77899b685e0 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:06.771750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4fe38-bc9b-4e9b-9469-ec7c8fcc9ffb None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:06.850246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55ae41-a25b-4cb9-ad0f-deee34d58303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:06.851197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55ae41-a25b-4cb9-ad0f-deee34d58303 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:06.851481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55ae41-a25b-4cb9-ad0f-deee34d58303 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:06.851876 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55ae41-a25b-4cb9-ad0f-deee34d58303 None admin] PATH_INFO: `/v3/users/efae8da899ef465d9872282b2bb91fef` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:06.855362 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55ae41-a25b-4cb9-ad0f-deee34d58303 None admin] RBAC: Authorizing `identity:get_user(user_id=efae8da899ef465d9872282b2bb91fef)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:06.857158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55ae41-a25b-4cb9-ad0f-deee34d58303 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:06.859325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 55/106] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:06 2022] GET /identity/v3/users/efae8da899ef465d9872282b2bb91fef => generated 247 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:06.877748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55ae41-a25b-4cb9-ad0f-deee34d58303 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:06.989778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:07.065286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4fe38-bc9b-4e9b-9469-ec7c8fcc9ffb None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:07.139896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5527be-add2-4395-8c10-b5735565b145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:07.141507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5527be-add2-4395-8c10-b5735565b145 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:07.142300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5527be-add2-4395-8c10-b5735565b145 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:07.143092 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5527be-add2-4395-8c10-b5735565b145 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:07.148426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55ae41-a25b-4cb9-ad0f-deee34d58303 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:07.174106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5527be-add2-4395-8c10-b5735565b145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:07.189821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf39cb6-5bef-4ec0-8647-8f55ebf7a521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:07.190494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf39cb6-5bef-4ec0-8647-8f55ebf7a521 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:07.190963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf39cb6-5bef-4ec0-8647-8f55ebf7a521 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:07.191315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf39cb6-5bef-4ec0-8647-8f55ebf7a521 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:07.193241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf39cb6-5bef-4ec0-8647-8f55ebf7a521 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:07.195363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf39cb6-5bef-4ec0-8647-8f55ebf7a521 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:07.197290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 56/107] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:06 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:07.198204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf39cb6-5bef-4ec0-8647-8f55ebf7a521 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-8f5527be-add2-4395-8c10-b5735565b145 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:07.204248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:07.205888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:07.207371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 52/108] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:07.215658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5527be-add2-4395-8c10-b5735565b145 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:07.312263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:07.366791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf39cb6-5bef-4ec0-8647-8f55ebf7a521 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:07.388221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:07.414269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7bafe-d005-4022-812f-f88f44ca69a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:07.414963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7bafe-d005-4022-812f-f88f44ca69a3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:07.415258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7bafe-d005-4022-812f-f88f44ca69a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:07.415578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7bafe-d005-4022-812f-f88f44ca69a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:07.429252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7bafe-d005-4022-812f-f88f44ca69a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-0eb7bafe-d005-4022-812f-f88f44ca69a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:07.436335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:07.438045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 57/109] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:05:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:07.439748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7bafe-d005-4022-812f-f88f44ca69a3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:07.610225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5527be-add2-4395-8c10-b5735565b145 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:07.706773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7bafe-d005-4022-812f-f88f44ca69a3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:07.707180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6fa70-8b02-4e73-9f3a-5ac58d3cada0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:07.708599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6fa70-8b02-4e73-9f3a-5ac58d3cada0 None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:07.708997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6fa70-8b02-4e73-9f3a-5ac58d3cada0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:07.709291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6fa70-8b02-4e73-9f3a-5ac58d3cada0 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/users/efae8da899ef465d9872282b2bb91fef/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:07.724855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6fa70-8b02-4e73-9f3a-5ac58d3cada0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfc85401ad649f0811b6b4ed33a5005, user_id=efae8da899ef465d9872282b2bb91fef, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:07.726520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd6fa70-8b02-4e73-9f3a-5ac58d3cada0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:07.761579 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176e6c4-fecd-486d-b9fe-85b952033d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:07.762241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176e6c4-fecd-486d-b9fe-85b952033d44 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:07.762710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176e6c4-fecd-486d-b9fe-85b952033d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:07.763007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176e6c4-fecd-486d-b9fe-85b952033d44 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:07.763866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176e6c4-fecd-486d-b9fe-85b952033d44 None admin] RBAC: Adding query filter params (name=member) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:07.764180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176e6c4-fecd-486d-b9fe-85b952033d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:07.765779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a176e6c4-fecd-486d-b9fe-85b952033d44 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:07.776681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 58/110] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:07.777694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176e6c4-fecd-486d-b9fe-85b952033d44 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:07.820755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 53/111] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:05:07 2022] PUT /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/users/efae8da899ef465d9872282b2bb91fef/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:05:07.821948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6fa70-8b02-4e73-9f3a-5ac58d3cada0 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:07.852956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:08.079118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176e6c4-fecd-486d-b9fe-85b952033d44 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:08.101857 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:08.130601 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6fa70-8b02-4e73-9f3a-5ac58d3cada0 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:08.165416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c54754-7da9-41bc-ace4-812c335bb11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:08.166220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54754-7da9-41bc-ace4-812c335bb11b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.166511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54754-7da9-41bc-ace4-812c335bb11b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.166793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c54754-7da9-41bc-ace4-812c335bb11b None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.167579 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c54754-7da9-41bc-ace4-812c335bb11b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:08.167986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c54754-7da9-41bc-ace4-812c335bb11b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:08.169648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c54754-7da9-41bc-ace4-812c335bb11b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:08.191424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 59/112] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:05:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:08.193621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fcbae-2506-417f-b94a-012ae33ec751 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.194803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fcbae-2506-417f-b94a-012ae33ec751 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.196542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fcbae-2506-417f-b94a-012ae33ec751 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.199965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 60/113] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:08.201354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fcbae-2506-417f-b94a-012ae33ec751 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:08.211404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928665f-e126-4a35-92f1-7a19aff25a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:08.212700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928665f-e126-4a35-92f1-7a19aff25a57 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.213059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928665f-e126-4a35-92f1-7a19aff25a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.213566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928665f-e126-4a35-92f1-7a19aff25a57 None admin] PATH_INFO: `/v3/users/37677eab531b4f658648830316674597` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.217064 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928665f-e126-4a35-92f1-7a19aff25a57 None admin] RBAC: Authorizing `identity:get_user(user_id=37677eab531b4f658648830316674597)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:08.219953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928665f-e126-4a35-92f1-7a19aff25a57 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:08.223017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 54/114] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:07 2022] GET /identity/v3/users/37677eab531b4f658648830316674597 => generated 275 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:08.225883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce1e5-75a6-4a87-8a64-e2a95d315e54 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.226363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce1e5-75a6-4a87-8a64-e2a95d315e54 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.226865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ce1e5-75a6-4a87-8a64-e2a95d315e54 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.229688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 55/115] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:08.236102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859edb2-938b-4fd2-9bf4-dc59cb867fe5 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.236400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859edb2-938b-4fd2-9bf4-dc59cb867fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.236700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859edb2-938b-4fd2-9bf4-dc59cb867fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.262879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-3859edb2-938b-4fd2-9bf4-dc59cb867fe5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:08.265015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3859edb2-938b-4fd2-9bf4-dc59cb867fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:08.353534 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:08.362734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fcbae-2506-417f-b94a-012ae33ec751 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:08.388507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 56/116] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:08.390835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ff36-40e6-4723-962b-d6a97653fae0 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.391346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ff36-40e6-4723-962b-d6a97653fae0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.391881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591ff36-40e6-4723-962b-d6a97653fae0 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.394410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 57/117] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:08.395765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591ff36-40e6-4723-962b-d6a97653fae0 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:08.422266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb1414-fa83-4a1a-bd4f-fed1ccd27970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:08.423375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb1414-fa83-4a1a-bd4f-fed1ccd27970 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.424022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb1414-fa83-4a1a-bd4f-fed1ccd27970 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.425123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb1414-fa83-4a1a-bd4f-fed1ccd27970 None admin] PATH_INFO: `/v3/users/37677eab531b4f658648830316674597` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.428520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb1414-fa83-4a1a-bd4f-fed1ccd27970 None admin] RBAC: Authorizing `identity:get_user(user_id=37677eab531b4f658648830316674597)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:08.430891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb1414-fa83-4a1a-bd4f-fed1ccd27970 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:08.433210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 61/118] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:08 2022] GET /identity/v3/users/37677eab531b4f658648830316674597 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:08.435107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc83e5-ca1c-4ff2-88bc-db3e73c6a208 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.435750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc83e5-ca1c-4ff2-88bc-db3e73c6a208 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.436241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc83e5-ca1c-4ff2-88bc-db3e73c6a208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.464607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-18fc83e5-ca1c-4ff2-88bc-db3e73c6a208 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:08.467002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-18fc83e5-ca1c-4ff2-88bc-db3e73c6a208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:08.556479 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 62/119] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:08.559481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17359f-e71c-4452-bd9d-0e016cd2a18b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.560054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17359f-e71c-4452-bd9d-0e016cd2a18b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.560540 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac17359f-e71c-4452-bd9d-0e016cd2a18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.571481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:08.583346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ac17359f-e71c-4452-bd9d-0e016cd2a18b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:08.586972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ac17359f-e71c-4452-bd9d-0e016cd2a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:08.620683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591ff36-40e6-4723-962b-d6a97653fae0 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:08.678379 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 63/120] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:08.680145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17359f-e71c-4452-bd9d-0e016cd2a18b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:08.728160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:08.762883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9f9b1-66a3-4560-bcbd-040ccda7b111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:08.764038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9f9b1-66a3-4560-bcbd-040ccda7b111 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.764551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9f9b1-66a3-4560-bcbd-040ccda7b111 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.765046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9f9b1-66a3-4560-bcbd-040ccda7b111 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.773055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9f9b1-66a3-4560-bcbd-040ccda7b111 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:08.776041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9f9b1-66a3-4560-bcbd-040ccda7b111 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:08.779797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 58/121] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:08 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:08.799323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9f9b1-66a3-4560-bcbd-040ccda7b111 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:08.824308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac17359f-e71c-4452-bd9d-0e016cd2a18b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:08.874442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1558c-f958-44bf-98af-4e51b21a8755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:08.875380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1558c-f958-44bf-98af-4e51b21a8755 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:08.875834 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1558c-f958-44bf-98af-4e51b21a8755 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:08.876343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1558c-f958-44bf-98af-4e51b21a8755 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:08.877754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1558c-f958-44bf-98af-4e51b21a8755 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:08.880278 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1558c-f958-44bf-98af-4e51b21a8755 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:08.882977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 64/122] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:08 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:08.893456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1558c-f958-44bf-98af-4e51b21a8755 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:09.039356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:09.043679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1558c-f958-44bf-98af-4e51b21a8755 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:09.137381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888130ba-1ce8-4135-95f1-ab55b244817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:09.138236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888130ba-1ce8-4135-95f1-ab55b244817e None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:09.138971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888130ba-1ce8-4135-95f1-ab55b244817e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:09.140527 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888130ba-1ce8-4135-95f1-ab55b244817e None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/users/37677eab531b4f658648830316674597/roles/b9c95d84eaca4444b2c7b8b5eed61a59` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:09.145236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888130ba-1ce8-4135-95f1-ab55b244817e None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfc85401ad649f0811b6b4ed33a5005, user_id=37677eab531b4f658648830316674597, role_id=b9c95d84eaca4444b2c7b8b5eed61a59)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:09.147184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888130ba-1ce8-4135-95f1-ab55b244817e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:09.161033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9f9b1-66a3-4560-bcbd-040ccda7b111 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:09.208838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129f718-2dc6-4fb4-b6a8-2964bf4ba4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:09.209754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129f718-2dc6-4fb4-b6a8-2964bf4ba4e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:09.210116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129f718-2dc6-4fb4-b6a8-2964bf4ba4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:09.210500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129f718-2dc6-4fb4-b6a8-2964bf4ba4e0 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/users/37677eab531b4f658648830316674597/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:09.212943 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129f718-2dc6-4fb4-b6a8-2964bf4ba4e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfc85401ad649f0811b6b4ed33a5005, user_id=37677eab531b4f658648830316674597, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:09.214864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129f718-2dc6-4fb4-b6a8-2964bf4ba4e0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:09.275066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 65/123] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:05:08 2022] PUT /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/users/37677eab531b4f658648830316674597/roles/b9c95d84eaca4444b2c7b8b5eed61a59 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:05:09.283810 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888130ba-1ce8-4135-95f1-ab55b244817e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:09.307643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 59/124] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:05:08 2022] PUT /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/users/37677eab531b4f658648830316674597/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:05:09.308872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129f718-2dc6-4fb4-b6a8-2964bf4ba4e0 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:09.366932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:09.574005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888130ba-1ce8-4135-95f1-ab55b244817e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:09.626101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e859a-a4c8-491f-b442-6b933db3033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:09.627475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e859a-a4c8-491f-b442-6b933db3033a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:09.628183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e859a-a4c8-491f-b442-6b933db3033a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:09.628743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e859a-a4c8-491f-b442-6b933db3033a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:09.642528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e859a-a4c8-491f-b442-6b933db3033a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:09.647347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129f718-2dc6-4fb4-b6a8-2964bf4ba4e0 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-463e859a-a4c8-491f-b442-6b933db3033a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:09.652001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:09.653147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:09.653927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 66/125] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:09.657329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e859a-a4c8-491f-b442-6b933db3033a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:09.682185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:09.740650 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79602084-bdcf-4775-b01f-cb2ddfba2168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:09.741752 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602084-bdcf-4775-b01f-cb2ddfba2168 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:09.742225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602084-bdcf-4775-b01f-cb2ddfba2168 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:09.742745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602084-bdcf-4775-b01f-cb2ddfba2168 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:09.764238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79602084-bdcf-4775-b01f-cb2ddfba2168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-79602084-bdcf-4775-b01f-cb2ddfba2168 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:09.773735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:09.776267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 60/126] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 467 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:09.778273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79602084-bdcf-4775-b01f-cb2ddfba2168 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:09.945656 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:09.963987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e859a-a4c8-491f-b442-6b933db3033a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:09.999447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79602084-bdcf-4775-b01f-cb2ddfba2168 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:10.072332 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70749db-cf41-4b4f-a382-fedf6e0cdc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:10.073202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70749db-cf41-4b4f-a382-fedf6e0cdc76 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:10.073603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70749db-cf41-4b4f-a382-fedf6e0cdc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:10.073991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70749db-cf41-4b4f-a382-fedf6e0cdc76 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:10.074992 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70749db-cf41-4b4f-a382-fedf6e0cdc76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:10.075438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70749db-cf41-4b4f-a382-fedf6e0cdc76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:10.078960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70749db-cf41-4b4f-a382-fedf6e0cdc76 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:10.097096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 61/127] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:10.099057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70749db-cf41-4b4f-a382-fedf6e0cdc76 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:10.120783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2de8ab-d379-4a45-926c-7d1dac6009b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:10.121845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2de8ab-d379-4a45-926c-7d1dac6009b6 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:10.122319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2de8ab-d379-4a45-926c-7d1dac6009b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:10.122809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2de8ab-d379-4a45-926c-7d1dac6009b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:10.128944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2de8ab-d379-4a45-926c-7d1dac6009b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:10.132357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2de8ab-d379-4a45-926c-7d1dac6009b6 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:10.136024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 67/128] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:05:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:10.137413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2de8ab-d379-4a45-926c-7d1dac6009b6 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:10.347060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70749db-cf41-4b4f-a382-fedf6e0cdc76 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:10.348358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:10.419703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2de8ab-d379-4a45-926c-7d1dac6009b6 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:10.445781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd99a75-f6ef-4cfb-81e0-996ad40520a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:10.447121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99a75-f6ef-4cfb-81e0-996ad40520a9 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:10.447819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99a75-f6ef-4cfb-81e0-996ad40520a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:10.448589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99a75-f6ef-4cfb-81e0-996ad40520a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:10.450077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd99a75-f6ef-4cfb-81e0-996ad40520a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:10.450883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd99a75-f6ef-4cfb-81e0-996ad40520a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:10.453498 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd99a75-f6ef-4cfb-81e0-996ad40520a9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:10.458220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:10.479594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 62/129] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:10.479594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd99a75-f6ef-4cfb-81e0-996ad40520a9 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:10.526783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391015b-d5e4-4065-bdba-acd5f95375a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:10.527823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391015b-d5e4-4065-bdba-acd5f95375a3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:10.529270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391015b-d5e4-4065-bdba-acd5f95375a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:10.529810 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391015b-d5e4-4065-bdba-acd5f95375a3 None admin] PATH_INFO: `/v3/users/fc34bd37fe314414a940e403c375234c` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:10.554967 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391015b-d5e4-4065-bdba-acd5f95375a3 None admin] RBAC: Authorizing `identity:get_user(user_id=fc34bd37fe314414a940e403c375234c)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:10.557339 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0391015b-d5e4-4065-bdba-acd5f95375a3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:10.559569 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 68/130] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:10 2022] GET /identity/v3/users/fc34bd37fe314414a940e403c375234c => generated 289 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:10.561161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391015b-d5e4-4065-bdba-acd5f95375a3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:10.802909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391015b-d5e4-4065-bdba-acd5f95375a3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:10.816509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd99a75-f6ef-4cfb-81e0-996ad40520a9 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:10.895959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa2d08-dee7-4c5b-877d-ffbcdc9ba355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:10.896926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa2d08-dee7-4c5b-877d-ffbcdc9ba355 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:10.897327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa2d08-dee7-4c5b-877d-ffbcdc9ba355 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:10.897719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa2d08-dee7-4c5b-877d-ffbcdc9ba355 None admin] PATH_INFO: `/v3/users/37677eab531b4f658648830316674597` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:10.901065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa2d08-dee7-4c5b-877d-ffbcdc9ba355 None admin] RBAC: Authorizing `identity:get_user(user_id=37677eab531b4f658648830316674597)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:10.903381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa2d08-dee7-4c5b-877d-ffbcdc9ba355 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:10.906065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 69/131] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:10 2022] GET /identity/v3/users/37677eab531b4f658648830316674597 => generated 275 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:10.907311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa2d08-dee7-4c5b-877d-ffbcdc9ba355 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:10.914994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad40e40-f4f7-4e09-9c08-7877e5463ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:10.915905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad40e40-f4f7-4e09-9c08-7877e5463ea8 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:10.916306 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad40e40-f4f7-4e09-9c08-7877e5463ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:10.916697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad40e40-f4f7-4e09-9c08-7877e5463ea8 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:10.917670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad40e40-f4f7-4e09-9c08-7877e5463ea8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:10.919907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad40e40-f4f7-4e09-9c08-7877e5463ea8 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:10.932049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:11.067918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 63/132] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:05:10 2022] POST /identity/v3/projects => generated 302 bytes in 590 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:05:11.069031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad40e40-f4f7-4e09-9c08-7877e5463ea8 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:11.146922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa2d08-dee7-4c5b-877d-ffbcdc9ba355 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:11.209441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:11.230337 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28176f-7fbc-41a3-9082-fdd7dec5c7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:11.231085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28176f-7fbc-41a3-9082-fdd7dec5c7fa None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:11.231390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28176f-7fbc-41a3-9082-fdd7dec5c7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:11.231836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28176f-7fbc-41a3-9082-fdd7dec5c7fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:11.384467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28176f-7fbc-41a3-9082-fdd7dec5c7fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:11.386786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28176f-7fbc-41a3-9082-fdd7dec5c7fa None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:11.391965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:11.394472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad40e40-f4f7-4e09-9c08-7877e5463ea8 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:11.531397 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181de7f-ad8d-4316-8709-e5513ee7ad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:11.532896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181de7f-ad8d-4316-8709-e5513ee7ad36 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:11.533403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181de7f-ad8d-4316-8709-e5513ee7ad36 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:11.533889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181de7f-ad8d-4316-8709-e5513ee7ad36 None admin] PATH_INFO: `/v3/projects/f2b4f9336b334662b06b49fda57b2fd1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:11.541084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181de7f-ad8d-4316-8709-e5513ee7ad36 None admin] RBAC: Authorizing `identity:get_project(project_id=f2b4f9336b334662b06b49fda57b2fd1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:11.544103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181de7f-ad8d-4316-8709-e5513ee7ad36 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:11.550061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 64/133] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:11 2022] GET /identity/v3/projects/f2b4f9336b334662b06b49fda57b2fd1 => generated 312 bytes in 479 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:11.561208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181de7f-ad8d-4316-8709-e5513ee7ad36 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:11.650031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 70/134] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 743 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:11.656548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28176f-7fbc-41a3-9082-fdd7dec5c7fa None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:11.820973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181de7f-ad8d-4316-8709-e5513ee7ad36 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:11.864773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:11.925122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400acac-b70e-4970-be75-669e55714b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:11.925945 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400acac-b70e-4970-be75-669e55714b83 None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:11.926321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400acac-b70e-4970-be75-669e55714b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:11.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400acac-b70e-4970-be75-669e55714b83 None admin] PATH_INFO: `/v3/projects/f2b4f9336b334662b06b49fda57b2fd1/users/37677eab531b4f658648830316674597/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:11.934111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400acac-b70e-4970-be75-669e55714b83 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2b4f9336b334662b06b49fda57b2fd1, user_id=37677eab531b4f658648830316674597, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:11.936063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400acac-b70e-4970-be75-669e55714b83 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:11.965290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28176f-7fbc-41a3-9082-fdd7dec5c7fa None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:11.968593 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:12.028071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa9516-a60a-4ac0-b235-91d90e779038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:12.029056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa9516-a60a-4ac0-b235-91d90e779038 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:12.029487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa9516-a60a-4ac0-b235-91d90e779038 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:12.029959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa9516-a60a-4ac0-b235-91d90e779038 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:12.032770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa9516-a60a-4ac0-b235-91d90e779038 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:12.035307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa9516-a60a-4ac0-b235-91d90e779038 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:12.038791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 71/135] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:11 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:12.047589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa9516-a60a-4ac0-b235-91d90e779038 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:12.068082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 65/136] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:05:11 2022] PUT /identity/v3/projects/f2b4f9336b334662b06b49fda57b2fd1/users/37677eab531b4f658648830316674597/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:05:12.216509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa9516-a60a-4ac0-b235-91d90e779038 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:12.275460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f8045-025f-4739-99b3-fb9490081553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:12.276368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8045-025f-4739-99b3-fb9490081553 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:12.276729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8045-025f-4739-99b3-fb9490081553 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:12.277020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f8045-025f-4739-99b3-fb9490081553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:12.277824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f8045-025f-4739-99b3-fb9490081553 None admin] RBAC: Adding query filter params (role.id=558d573b58fa4ba59d56e645fd5a0000, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005, user.id=fc34bd37fe314414a940e403c375234c) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:12.278324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f8045-025f-4739-99b3-fb9490081553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:12.279853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f8045-025f-4739-99b3-fb9490081553 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:12.288957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 72/137] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:12 2022] GET /identity/v3/role_assignments?user.id=fc34bd37fe314414a940e403c375234c&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=558d573b58fa4ba59d56e645fd5a0000 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:12.342335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4437f0-2711-49d1-bc19-76fbfaf59f9d None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:12.342721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4437f0-2711-49d1-bc19-76fbfaf59f9d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:12.342930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4437f0-2711-49d1-bc19-76fbfaf59f9d None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:12.344869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 66/138] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:12.360704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb213d08-5eb8-4562-9396-1b4bc4947820 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:12.361106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb213d08-5eb8-4562-9396-1b4bc4947820 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:12.361338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb213d08-5eb8-4562-9396-1b4bc4947820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:12.392810 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-bb213d08-5eb8-4562-9396-1b4bc4947820 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:12.394074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bb213d08-5eb8-4562-9396-1b4bc4947820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:12.439979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:12.495300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 73/139] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:13.082465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4437f0-2711-49d1-bc19-76fbfaf59f9d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:13.249509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4437f0-2711-49d1-bc19-76fbfaf59f9d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:13.301327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a310022-28e9-4804-acb4-3b8991afdf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:13.302038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a310022-28e9-4804-acb4-3b8991afdf44 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:13.302323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a310022-28e9-4804-acb4-3b8991afdf44 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:13.302671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a310022-28e9-4804-acb4-3b8991afdf44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:13.319420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a310022-28e9-4804-acb4-3b8991afdf44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-0a310022-28e9-4804-acb4-3b8991afdf44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:13.325410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:13.327061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:13.328603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 67/140] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:05:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:05:13.340218 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb213d08-5eb8-4562-9396-1b4bc4947820 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:13.411594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890e6cd-5e10-4f22-868d-e06961629236 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:13.411594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890e6cd-5e10-4f22-868d-e06961629236 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:13.411594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890e6cd-5e10-4f22-868d-e06961629236 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:13.415216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 68/141] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:13.425165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c657e-8d9f-4b75-8c6a-7cca5b3eee35 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:13.426055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c657e-8d9f-4b75-8c6a-7cca5b3eee35 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:13.426846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c657e-8d9f-4b75-8c6a-7cca5b3eee35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:13.468516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-688c657e-8d9f-4b75-8c6a-7cca5b3eee35 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:13.471208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-688c657e-8d9f-4b75-8c6a-7cca5b3eee35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:13.532126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:13.577794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb213d08-5eb8-4562-9396-1b4bc4947820 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:13.630412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c5461-741a-48a6-aa91-4342e1448e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:13.631093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c5461-741a-48a6-aa91-4342e1448e85 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:13.632081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c5461-741a-48a6-aa91-4342e1448e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:13.632081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c5461-741a-48a6-aa91-4342e1448e85 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:13.632725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c5461-741a-48a6-aa91-4342e1448e85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:13.633083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c5461-741a-48a6-aa91-4342e1448e85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:13.636612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c5461-741a-48a6-aa91-4342e1448e85 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:13.646491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 74/142] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:13.652587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c5461-741a-48a6-aa91-4342e1448e85 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:13.654838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 69/143] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:13.698355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:13.812306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c5461-741a-48a6-aa91-4342e1448e85 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:13.868971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ccc5a-6d5c-4c19-9292-ba36699d2fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:13.869959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ccc5a-6d5c-4c19-9292-ba36699d2fcb None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:13.870276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ccc5a-6d5c-4c19-9292-ba36699d2fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:13.870685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ccc5a-6d5c-4c19-9292-ba36699d2fcb None admin] PATH_INFO: `/v3/users/efae8da899ef465d9872282b2bb91fef` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:13.872424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ccc5a-6d5c-4c19-9292-ba36699d2fcb None admin] RBAC: Authorizing `identity:get_user(user_id=efae8da899ef465d9872282b2bb91fef)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:13.874239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ccc5a-6d5c-4c19-9292-ba36699d2fcb None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:13.882618 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 75/144] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:13 2022] GET /identity/v3/users/efae8da899ef465d9872282b2bb91fef => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:13.884870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c657e-8d9f-4b75-8c6a-7cca5b3eee35 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:14.103697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c657e-8d9f-4b75-8c6a-7cca5b3eee35 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:14.156288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffef16-3f74-4d24-81a6-99022309086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:14.156832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffef16-3f74-4d24-81a6-99022309086e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:14.157121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffef16-3f74-4d24-81a6-99022309086e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:14.157400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffef16-3f74-4d24-81a6-99022309086e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:14.222393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:14.241581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2934d1d-5aee-43e4-801d-0d5816a4eaab None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:14.242003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2934d1d-5aee-43e4-801d-0d5816a4eaab None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:14.242458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2934d1d-5aee-43e4-801d-0d5816a4eaab None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:14.244021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 76/145] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:05:14.257208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d719a-c3a6-4a16-a154-9ce7a7a6f2b5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:14.257475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d719a-c3a6-4a16-a154-9ce7a7a6f2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:14.257804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d719a-c3a6-4a16-a154-9ce7a7a6f2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:14.288535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-db9d719a-c3a6-4a16-a154-9ce7a7a6f2b5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:14.291605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-db9d719a-c3a6-4a16-a154-9ce7a7a6f2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:14.316118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffef16-3f74-4d24-81a6-99022309086e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:14.317836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ffef16-3f74-4d24-81a6-99022309086e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:14.508285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 77/146] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:14.509793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d719a-c3a6-4a16-a154-9ce7a7a6f2b5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:14.562519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:14.622846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 70/147] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:14.628919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffef16-3f74-4d24-81a6-99022309086e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:14.743627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d719a-c3a6-4a16-a154-9ce7a7a6f2b5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:14.849894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:14.856092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b02ba-6932-4583-afbb-62e6694ef250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:14.857641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b02ba-6932-4583-afbb-62e6694ef250 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:14.861353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b02ba-6932-4583-afbb-62e6694ef250 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:14.861353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b02ba-6932-4583-afbb-62e6694ef250 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:14.882010 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b02ba-6932-4583-afbb-62e6694ef250 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-814b02ba-6932-4583-afbb-62e6694ef250 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:14.889001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:14.889969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:05:14.890733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:14.893780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 78/148] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:05:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:14.899390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b02ba-6932-4583-afbb-62e6694ef250 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:14.933950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffef16-3f74-4d24-81a6-99022309086e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:15.058812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c78a4-ff01-471a-8722-62d061b327dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:15.059922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c78a4-ff01-471a-8722-62d061b327dd None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:15.060406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c78a4-ff01-471a-8722-62d061b327dd None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:15.060905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c78a4-ff01-471a-8722-62d061b327dd None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:15.069176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c78a4-ff01-471a-8722-62d061b327dd None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:15.071266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c78a4-ff01-471a-8722-62d061b327dd None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:15.073666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 71/149] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:14 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 445 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:15.097311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c78a4-ff01-471a-8722-62d061b327dd None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:15.145762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:15.182071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b02ba-6932-4583-afbb-62e6694ef250 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:15.264942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8008a-9d80-4c1b-864e-1f7577ed96f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:15.266035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8008a-9d80-4c1b-864e-1f7577ed96f8 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:15.266550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8008a-9d80-4c1b-864e-1f7577ed96f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:15.267056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8008a-9d80-4c1b-864e-1f7577ed96f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:15.268413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8008a-9d80-4c1b-864e-1f7577ed96f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:15.269014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8008a-9d80-4c1b-864e-1f7577ed96f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:15.272782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8008a-9d80-4c1b-864e-1f7577ed96f8 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:15.291737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 79/150] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:05:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:15.293510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8008a-9d80-4c1b-864e-1f7577ed96f8 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:15.309675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c78a4-ff01-471a-8722-62d061b327dd None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:15.368671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617ac09-aad8-4cbd-94ec-cfdeb2115cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:15.369960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617ac09-aad8-4cbd-94ec-cfdeb2115cca None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:15.370587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617ac09-aad8-4cbd-94ec-cfdeb2115cca None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:15.371202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617ac09-aad8-4cbd-94ec-cfdeb2115cca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:15.372590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617ac09-aad8-4cbd-94ec-cfdeb2115cca None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005, user.id=efae8da899ef465d9872282b2bb91fef) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:15.373313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617ac09-aad8-4cbd-94ec-cfdeb2115cca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:15.375919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617ac09-aad8-4cbd-94ec-cfdeb2115cca None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:15.381510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:15.386952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 72/151] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:15 2022] GET /identity/v3/role_assignments?user.id=efae8da899ef465d9872282b2bb91fef&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:15.389235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330fd0a-b398-4e8c-a94c-4d8a926553cf None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:15.389746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330fd0a-b398-4e8c-a94c-4d8a926553cf None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:15.390226 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330fd0a-b398-4e8c-a94c-4d8a926553cf None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:15.392347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 73/152] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:15.399996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330fd0a-b398-4e8c-a94c-4d8a926553cf None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:15.548783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330fd0a-b398-4e8c-a94c-4d8a926553cf None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:15.552757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8008a-9d80-4c1b-864e-1f7577ed96f8 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:15.597862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b50987-a3cc-4811-a2f1-3111f41636f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:15.598988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b50987-a3cc-4811-a2f1-3111f41636f4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:15.598988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b50987-a3cc-4811-a2f1-3111f41636f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:15.599533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b50987-a3cc-4811-a2f1-3111f41636f4 None admin] PATH_INFO: `/v3/users/37677eab531b4f658648830316674597` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:15.601249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b50987-a3cc-4811-a2f1-3111f41636f4 None admin] RBAC: Authorizing `identity:get_user(user_id=37677eab531b4f658648830316674597)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:15.603086 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b50987-a3cc-4811-a2f1-3111f41636f4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:15.605776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 74/153] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:15 2022] GET /identity/v3/users/37677eab531b4f658648830316674597 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:15.607405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c47ca-d60c-4f85-bb1a-d47d4163441b None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:15.607844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c47ca-d60c-4f85-bb1a-d47d4163441b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:15.608215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c47ca-d60c-4f85-bb1a-d47d4163441b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:15.632314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-223c47ca-d60c-4f85-bb1a-d47d4163441b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:15.634174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-223c47ca-d60c-4f85-bb1a-d47d4163441b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:15.642412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c0ee3-9125-45d3-9126-debbcdd22406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:15.644050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c0ee3-9125-45d3-9126-debbcdd22406 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:15.644913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c0ee3-9125-45d3-9126-debbcdd22406 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:15.645695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c0ee3-9125-45d3-9126-debbcdd22406 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:15.672567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c0ee3-9125-45d3-9126-debbcdd22406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-475c0ee3-9125-45d3-9126-debbcdd22406 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:15.688051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:15.690395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:15.692128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:15.698161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 80/154] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:15.700124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c0ee3-9125-45d3-9126-debbcdd22406 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:15.727771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:15.788907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:15.797028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 75/155] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:15.798060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c47ca-d60c-4f85-bb1a-d47d4163441b None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:15.928732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c0ee3-9125-45d3-9126-debbcdd22406 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:15.986755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c021ea-b402-4899-924f-1f4c99eeee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:15.987641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c021ea-b402-4899-924f-1f4c99eeee0f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:15.987962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c021ea-b402-4899-924f-1f4c99eeee0f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:15.988236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c021ea-b402-4899-924f-1f4c99eeee0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:16.024898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c47ca-d60c-4f85-bb1a-d47d4163441b None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:16.081594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c021ea-b402-4899-924f-1f4c99eeee0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:16.083415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c021ea-b402-4899-924f-1f4c99eeee0f None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:16.085456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220223b-c4a8-41f7-b4da-62c01227087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:16.086381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220223b-c4a8-41f7-b4da-62c01227087e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:16.086770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220223b-c4a8-41f7-b4da-62c01227087e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:16.087201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220223b-c4a8-41f7-b4da-62c01227087e None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:16.088279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220223b-c4a8-41f7-b4da-62c01227087e None admin] RBAC: Adding query filter params (name=member) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:16.088703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220223b-c4a8-41f7-b4da-62c01227087e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:16.090812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220223b-c4a8-41f7-b4da-62c01227087e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:16.101547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:16.107217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 76/156] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:16.114648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a52ad-7f58-4fef-82bb-b5d149405720 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:16.118210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a52ad-7f58-4fef-82bb-b5d149405720 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:16.118210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a52ad-7f58-4fef-82bb-b5d149405720 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:16.121610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 77/157] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:16 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:16.122879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a52ad-7f58-4fef-82bb-b5d149405720 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:16.216441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:16.238053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 81/158] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:16.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0186f-1582-462c-9b1c-dfee28b5ddc4 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:16.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0186f-1582-462c-9b1c-dfee28b5ddc4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:16.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0186f-1582-462c-9b1c-dfee28b5ddc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:16.261926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b2c0186f-1582-462c-9b1c-dfee28b5ddc4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:16.263207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b2c0186f-1582-462c-9b1c-dfee28b5ddc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:16.299467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a52ad-7f58-4fef-82bb-b5d149405720 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:16.353695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 82/159] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:16.361825 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0186f-1582-462c-9b1c-dfee28b5ddc4 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:16.374254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3aebbb-6f14-4e6c-8aa6-abd834b975bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:16.376502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aebbb-6f14-4e6c-8aa6-abd834b975bc None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:16.377282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aebbb-6f14-4e6c-8aa6-abd834b975bc None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:16.377994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aebbb-6f14-4e6c-8aa6-abd834b975bc None admin] PATH_INFO: `/v3/users/37677eab531b4f658648830316674597` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:16.381387 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3aebbb-6f14-4e6c-8aa6-abd834b975bc None admin] RBAC: Authorizing `identity:get_user(user_id=37677eab531b4f658648830316674597)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:16.384595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3aebbb-6f14-4e6c-8aa6-abd834b975bc None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:16.388176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 78/160] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:16 2022] GET /identity/v3/users/37677eab531b4f658648830316674597 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:16.397438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3aebbb-6f14-4e6c-8aa6-abd834b975bc None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:16.458608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:16.586208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0186f-1582-462c-9b1c-dfee28b5ddc4 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:16.626766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3aebbb-6f14-4e6c-8aa6-abd834b975bc None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:16.654764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3fee7-34ca-4280-a138-cea2ef9b7d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:16.655496 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3fee7-34ca-4280-a138-cea2ef9b7d48 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:16.655830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3fee7-34ca-4280-a138-cea2ef9b7d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:16.656145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3fee7-34ca-4280-a138-cea2ef9b7d48 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:16.658081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3fee7-34ca-4280-a138-cea2ef9b7d48 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:16.663011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3fee7-34ca-4280-a138-cea2ef9b7d48 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:16.665376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 83/161] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:16 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:16.666447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3fee7-34ca-4280-a138-cea2ef9b7d48 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:16.715982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:16.728533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e85b24-7e4f-4c01-b65e-347a67513cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:16.729335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e85b24-7e4f-4c01-b65e-347a67513cbf None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:16.729617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e85b24-7e4f-4c01-b65e-347a67513cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:16.729893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e85b24-7e4f-4c01-b65e-347a67513cbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:16.808853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3fee7-34ca-4280-a138-cea2ef9b7d48 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:16.820080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:16.827314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e85b24-7e4f-4c01-b65e-347a67513cbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:16.827588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e85b24-7e4f-4c01-b65e-347a67513cbf None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:16.853225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8db25-d471-4590-ae58-fb3c322e71cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:16.853793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8db25-d471-4590-ae58-fb3c322e71cf None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:16.854073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8db25-d471-4590-ae58-fb3c322e71cf None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:16.854486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8db25-d471-4590-ae58-fb3c322e71cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:16.855744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8db25-d471-4590-ae58-fb3c322e71cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:16.857320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb8db25-d471-4590-ae58-fb3c322e71cf None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:16.859191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 84/162] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:05:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:16.860101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8db25-d471-4590-ae58-fb3c322e71cf None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:16.984566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 79/163] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 01:05:16.986259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e85b24-7e4f-4c01-b65e-347a67513cbf None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:17.011597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8db25-d471-4590-ae58-fb3c322e71cf None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:17.050136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:17.058686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fd54d-ed08-40bb-99cb-95a7d07b1aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:17.059377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fd54d-ed08-40bb-99cb-95a7d07b1aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:17.063925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fd54d-ed08-40bb-99cb-95a7d07b1aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:17.063925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fd54d-ed08-40bb-99cb-95a7d07b1aa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:17.063925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fd54d-ed08-40bb-99cb-95a7d07b1aa7 None admin] RBAC: Adding query filter params (role.id=b9c95d84eaca4444b2c7b8b5eed61a59, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005, user.id=37677eab531b4f658648830316674597) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:17.063925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fd54d-ed08-40bb-99cb-95a7d07b1aa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:17.063925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fd54d-ed08-40bb-99cb-95a7d07b1aa7 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:17.071741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 85/164] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:16 2022] GET /identity/v3/role_assignments?user.id=37677eab531b4f658648830316674597&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=b9c95d84eaca4444b2c7b8b5eed61a59 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:17.073818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20894475-62a8-44a1-a944-14bbf3487a7f None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:17.073818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20894475-62a8-44a1-a944-14bbf3487a7f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:17.074301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20894475-62a8-44a1-a944-14bbf3487a7f None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:17.075888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 86/165] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:17.076491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20894475-62a8-44a1-a944-14bbf3487a7f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:17.185555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:17.209139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e85b24-7e4f-4c01-b65e-347a67513cbf None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:17.282643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554efcba-489a-4c0b-8f75-9002d9855340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:17.283767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554efcba-489a-4c0b-8f75-9002d9855340 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:17.284249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554efcba-489a-4c0b-8f75-9002d9855340 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:17.284733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554efcba-489a-4c0b-8f75-9002d9855340 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:17.304248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554efcba-489a-4c0b-8f75-9002d9855340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-554efcba-489a-4c0b-8f75-9002d9855340 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:17.313961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:17.315882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 80/166] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:17.319510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554efcba-489a-4c0b-8f75-9002d9855340 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:17.326103 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20894475-62a8-44a1-a944-14bbf3487a7f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:17.403708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88d294-b09e-4570-bfb6-cb859169f6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:17.404398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88d294-b09e-4570-bfb6-cb859169f6c9 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:17.404685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88d294-b09e-4570-bfb6-cb859169f6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:17.404998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88d294-b09e-4570-bfb6-cb859169f6c9 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:17.406245 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88d294-b09e-4570-bfb6-cb859169f6c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:17.408412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88d294-b09e-4570-bfb6-cb859169f6c9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:17.479193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:17.499827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554efcba-489a-4c0b-8f75-9002d9855340 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:17.530782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 87/167] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:05:17 2022] POST /identity/v3/users => generated 283 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:05:17.540832 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e802c-e3f1-4727-8853-ec81e12d0354 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:17.541564 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e802c-e3f1-4727-8853-ec81e12d0354 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:17.541953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e802c-e3f1-4727-8853-ec81e12d0354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:17.546352 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e713b9-5895-493e-bd5d-aec7a79c9a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:17.547347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e713b9-5895-493e-bd5d-aec7a79c9a3c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:17.547881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e713b9-5895-493e-bd5d-aec7a79c9a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:17.548350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e713b9-5895-493e-bd5d-aec7a79c9a3c None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:17.550473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e713b9-5895-493e-bd5d-aec7a79c9a3c None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:17.553166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e713b9-5895-493e-bd5d-aec7a79c9a3c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:17.555756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 81/168] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:17 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:17.555756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e713b9-5895-493e-bd5d-aec7a79c9a3c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:17.586366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-de6e802c-e3f1-4727-8853-ec81e12d0354 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:17.588947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-de6e802c-e3f1-4727-8853-ec81e12d0354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:17.726581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e713b9-5895-493e-bd5d-aec7a79c9a3c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:17.736929 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 88/169] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:17.738276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e802c-e3f1-4727-8853-ec81e12d0354 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:17.787755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33bd96-81b4-4147-be6f-b2b36efd784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:17.788393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33bd96-81b4-4147-be6f-b2b36efd784f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:17.788766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33bd96-81b4-4147-be6f-b2b36efd784f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:17.789081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33bd96-81b4-4147-be6f-b2b36efd784f None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:17.789780 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33bd96-81b4-4147-be6f-b2b36efd784f None admin] RBAC: Adding query filter params (name=member) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:17.790142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33bd96-81b4-4147-be6f-b2b36efd784f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:17.794246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33bd96-81b4-4147-be6f-b2b36efd784f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:17.794465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:17.806840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 82/170] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:17.820325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33bd96-81b4-4147-be6f-b2b36efd784f None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:17.861169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:17.933375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e802c-e3f1-4727-8853-ec81e12d0354 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:17.990616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33bd96-81b4-4147-be6f-b2b36efd784f None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:18.012493 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55ed95-46bd-45de-8ffd-b9ec72cb0ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:18.013595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55ed95-46bd-45de-8ffd-b9ec72cb0ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:18.014099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55ed95-46bd-45de-8ffd-b9ec72cb0ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:18.014593 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55ed95-46bd-45de-8ffd-b9ec72cb0ac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:18.016092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55ed95-46bd-45de-8ffd-b9ec72cb0ac6 None admin] RBAC: Adding query filter params (role.id=e5a65ae2d0884e419cdbc70c9223eac2, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005, user.id=37677eab531b4f658648830316674597) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:18.016648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55ed95-46bd-45de-8ffd-b9ec72cb0ac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:18.020271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55ed95-46bd-45de-8ffd-b9ec72cb0ac6 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:18.031938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 89/171] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:17 2022] GET /identity/v3/role_assignments?user.id=37677eab531b4f658648830316674597&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:18.038338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f76199-dc15-4468-a435-84f9957afe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:18.039169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f76199-dc15-4468-a435-84f9957afe30 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:18.039664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f76199-dc15-4468-a435-84f9957afe30 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:18.040264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f76199-dc15-4468-a435-84f9957afe30 None admin] PATH_INFO: `/v3/users/37677eab531b4f658648830316674597` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:18.042224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f76199-dc15-4468-a435-84f9957afe30 None admin] RBAC: Authorizing `identity:get_user(user_id=37677eab531b4f658648830316674597)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:18.044096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f76199-dc15-4468-a435-84f9957afe30 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:18.046187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 83/172] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:17 2022] GET /identity/v3/users/37677eab531b4f658648830316674597 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:18.052998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55ed95-46bd-45de-8ffd-b9ec72cb0ac6 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:18.107621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f76199-dc15-4468-a435-84f9957afe30 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:18.223526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:18.254947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55ed95-46bd-45de-8ffd-b9ec72cb0ac6 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:18.269319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:18.312119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f76199-dc15-4468-a435-84f9957afe30 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:18.326774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c92f05-7909-446f-b727-811d4aa8a617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:18.327982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c92f05-7909-446f-b727-811d4aa8a617 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:18.328561 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c92f05-7909-446f-b727-811d4aa8a617 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:18.329050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c92f05-7909-446f-b727-811d4aa8a617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:18.385689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d79c23-e838-4857-b042-0c373d51f174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:18.386623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d79c23-e838-4857-b042-0c373d51f174 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:18.387054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d79c23-e838-4857-b042-0c373d51f174 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:18.388818 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d79c23-e838-4857-b042-0c373d51f174 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:18.412452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d79c23-e838-4857-b042-0c373d51f174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-31d79c23-e838-4857-b042-0c373d51f174 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:18.421381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:18.422553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:18.423331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 84/173] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:05:18.432887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d79c23-e838-4857-b042-0c373d51f174 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:18.459172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c92f05-7909-446f-b727-811d4aa8a617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:18.461818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c92f05-7909-446f-b727-811d4aa8a617 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:18.606219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:18.621367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 90/174] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:18.627590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c92f05-7909-446f-b727-811d4aa8a617 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:18.639269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d79c23-e838-4857-b042-0c373d51f174 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:18.718766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56369c7-9698-4aba-961c-2e11ca1d81be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:18.721112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56369c7-9698-4aba-961c-2e11ca1d81be None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:18.721704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56369c7-9698-4aba-961c-2e11ca1d81be None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:18.722197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56369c7-9698-4aba-961c-2e11ca1d81be None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:18.724184 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56369c7-9698-4aba-961c-2e11ca1d81be None admin] RBAC: Adding query filter params (name=reader) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:18.724713 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56369c7-9698-4aba-961c-2e11ca1d81be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:18.727409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56369c7-9698-4aba-961c-2e11ca1d81be None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:18.747172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 85/175] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:18.756061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56369c7-9698-4aba-961c-2e11ca1d81be None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:18.786481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:18.825079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c92f05-7909-446f-b727-811d4aa8a617 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:18.938216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc985b2-24f9-40db-9f70-ea104018036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:18.939210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc985b2-24f9-40db-9f70-ea104018036a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:18.939649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc985b2-24f9-40db-9f70-ea104018036a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:18.940064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc985b2-24f9-40db-9f70-ea104018036a None admin] PATH_INFO: `/v3/projects/f2b4f9336b334662b06b49fda57b2fd1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:18.943295 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc985b2-24f9-40db-9f70-ea104018036a None admin] RBAC: Authorizing `identity:get_project(project_id=f2b4f9336b334662b06b49fda57b2fd1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:18.945968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc985b2-24f9-40db-9f70-ea104018036a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:18.957991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 91/176] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:18 2022] GET /identity/v3/projects/f2b4f9336b334662b06b49fda57b2fd1 => generated 312 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:18.966031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc985b2-24f9-40db-9f70-ea104018036a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:19.059990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56369c7-9698-4aba-961c-2e11ca1d81be None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:19.156066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:19.165207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d11a3-7155-41d0-add3-0bc9d15e5e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:19.166281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d11a3-7155-41d0-add3-0bc9d15e5e1e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:19.166806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d11a3-7155-41d0-add3-0bc9d15e5e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:19.167290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d11a3-7155-41d0-add3-0bc9d15e5e1e None admin] PATH_INFO: `/v3/users/fc34bd37fe314414a940e403c375234c` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:19.172508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d11a3-7155-41d0-add3-0bc9d15e5e1e None admin] RBAC: Authorizing `identity:get_user(user_id=fc34bd37fe314414a940e403c375234c)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:19.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d11a3-7155-41d0-add3-0bc9d15e5e1e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:19.176624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 86/177] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:18 2022] GET /identity/v3/users/fc34bd37fe314414a940e403c375234c => generated 289 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:19.198233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d11a3-7155-41d0-add3-0bc9d15e5e1e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:19.222310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc985b2-24f9-40db-9f70-ea104018036a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:19.290705 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c519d-be5f-4d5f-93d2-4bc37a698680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:19.291364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c519d-be5f-4d5f-93d2-4bc37a698680 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:19.291824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c519d-be5f-4d5f-93d2-4bc37a698680 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:19.292113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c519d-be5f-4d5f-93d2-4bc37a698680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:19.292940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c519d-be5f-4d5f-93d2-4bc37a698680 None admin] RBAC: Adding query filter params (role.id=e5a65ae2d0884e419cdbc70c9223eac2, scope.project.id=f2b4f9336b334662b06b49fda57b2fd1, user.id=37677eab531b4f658648830316674597) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:19.293256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c519d-be5f-4d5f-93d2-4bc37a698680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:19.302235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c519d-be5f-4d5f-93d2-4bc37a698680 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:19.315397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 92/178] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:18 2022] GET /identity/v3/role_assignments?user.id=37677eab531b4f658648830316674597&scope.project.id=f2b4f9336b334662b06b49fda57b2fd1&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 608 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:19.372411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:19.418998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d11a3-7155-41d0-add3-0bc9d15e5e1e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:19.497655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b82491-8c19-438d-8588-c1c87883a075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:19.498686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82491-8c19-438d-8588-c1c87883a075 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:19.499187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82491-8c19-438d-8588-c1c87883a075 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:19.499709 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82491-8c19-438d-8588-c1c87883a075 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:19.501962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b82491-8c19-438d-8588-c1c87883a075 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:19.505028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b82491-8c19-438d-8588-c1c87883a075 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:19.515755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 87/179] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:19 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:19.518949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c519d-be5f-4d5f-93d2-4bc37a698680 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:19.734175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c519d-be5f-4d5f-93d2-4bc37a698680 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:19.779041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:19.806670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12589574-bd84-4597-bd5c-83fdee90cf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:19.807712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12589574-bd84-4597-bd5c-83fdee90cf98 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:19.808215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12589574-bd84-4597-bd5c-83fdee90cf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:19.808684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12589574-bd84-4597-bd5c-83fdee90cf98 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/users/fc34bd37fe314414a940e403c375234c/roles/558d573b58fa4ba59d56e645fd5a0000` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:19.815467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12589574-bd84-4597-bd5c-83fdee90cf98 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfc85401ad649f0811b6b4ed33a5005, user_id=fc34bd37fe314414a940e403c375234c, role_id=558d573b58fa4ba59d56e645fd5a0000)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:19.817815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12589574-bd84-4597-bd5c-83fdee90cf98 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:19.928286 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 93/180] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:05:19 2022] PUT /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/users/fc34bd37fe314414a940e403c375234c/roles/558d573b58fa4ba59d56e645fd5a0000 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:05:20.699035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70009ba-976a-455c-be38-7e5590f7c5c9 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:20.699877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70009ba-976a-455c-be38-7e5590f7c5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:20.699877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70009ba-976a-455c-be38-7e5590f7c5c9 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:20.701661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 88/181] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:20.709406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93eea5-fd24-42e8-b077-cf883ce7aee5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:20.709924 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93eea5-fd24-42e8-b077-cf883ce7aee5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:20.710426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93eea5-fd24-42e8-b077-cf883ce7aee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:20.746334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6a93eea5-fd24-42e8-b077-cf883ce7aee5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:20.748645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6a93eea5-fd24-42e8-b077-cf883ce7aee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:20.845999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 94/182] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:21.167989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70009ba-976a-455c-be38-7e5590f7c5c9 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:21.358389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70009ba-976a-455c-be38-7e5590f7c5c9 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:21.425933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2220b8b-ecde-4f9e-bbe8-dd9fac0283cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:21.426890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220b8b-ecde-4f9e-bbe8-dd9fac0283cf None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:21.427377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220b8b-ecde-4f9e-bbe8-dd9fac0283cf None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:21.427865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2220b8b-ecde-4f9e-bbe8-dd9fac0283cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:21.450690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2220b8b-ecde-4f9e-bbe8-dd9fac0283cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-e2220b8b-ecde-4f9e-bbe8-dd9fac0283cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:21.461509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:21.463254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:21.464802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 89/183] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:05:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:05:21.472776 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93eea5-fd24-42e8-b077-cf883ce7aee5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:21.603443 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3b576-faea-4609-9bb0-98a31fc928ae None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:21.605876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3b576-faea-4609-9bb0-98a31fc928ae None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:21.605876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3b576-faea-4609-9bb0-98a31fc928ae None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:21.609171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 90/184] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:05:21.617065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1a11e-22c4-44b4-b124-9b571d66c782 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:21.617631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1a11e-22c4-44b4-b124-9b571d66c782 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:21.618161 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1a11e-22c4-44b4-b124-9b571d66c782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:21.636145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:21.655431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-40f1a11e-22c4-44b4-b124-9b571d66c782 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:21.657664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-40f1a11e-22c4-44b4-b124-9b571d66c782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:21.663959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93eea5-fd24-42e8-b077-cf883ce7aee5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:21.682046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:21.715926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6de67-45b5-4bab-8496-e0da996b477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:21.718007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6de67-45b5-4bab-8496-e0da996b477d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:21.718589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6de67-45b5-4bab-8496-e0da996b477d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:21.719298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6de67-45b5-4bab-8496-e0da996b477d None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:21.720839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6de67-45b5-4bab-8496-e0da996b477d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:21.721737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6de67-45b5-4bab-8496-e0da996b477d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:21.724364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6de67-45b5-4bab-8496-e0da996b477d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:21.736490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 95/185] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:21.744706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6de67-45b5-4bab-8496-e0da996b477d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:21.800682 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 91/186] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:21.874353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6de67-45b5-4bab-8496-e0da996b477d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:21.917114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67c8af-b594-4b76-8f49-2ae697358345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:21.917772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67c8af-b594-4b76-8f49-2ae697358345 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:21.918048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67c8af-b594-4b76-8f49-2ae697358345 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:21.918323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67c8af-b594-4b76-8f49-2ae697358345 None admin] PATH_INFO: `/v3/users/5bd0770700d54763aa15987cb7f1a444` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:21.935680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67c8af-b594-4b76-8f49-2ae697358345 None admin] RBAC: Authorizing `identity:get_user(user_id=5bd0770700d54763aa15987cb7f1a444)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:21.937175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67c8af-b594-4b76-8f49-2ae697358345 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:21.939338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 96/187] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:21 2022] GET /identity/v3/users/5bd0770700d54763aa15987cb7f1a444 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:21.946261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1a11e-22c4-44b4-b124-9b571d66c782 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:21.950307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf2f4e-623b-4184-9993-863d513d6c2c None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:21.950660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf2f4e-623b-4184-9993-863d513d6c2c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:21.950893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf2f4e-623b-4184-9993-863d513d6c2c None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:21.952446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 97/188] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:21.958796 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af40f-1904-48c2-811e-96240ee0fc25 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:21.959040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af40f-1904-48c2-811e-96240ee0fc25 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:21.959300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3af40f-1904-48c2-811e-96240ee0fc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:21.980877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-cf3af40f-1904-48c2-811e-96240ee0fc25 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:21.982959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-cf3af40f-1904-48c2-811e-96240ee0fc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:22.015156 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:22.094100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 98/189] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:22.160055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1a11e-22c4-44b4-b124-9b571d66c782 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:22.230492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0632e-dd93-4928-89a9-a7aca813490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:22.231487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0632e-dd93-4928-89a9-a7aca813490b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:22.231999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0632e-dd93-4928-89a9-a7aca813490b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:22.232485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0632e-dd93-4928-89a9-a7aca813490b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:22.236088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:22.349651 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3af40f-1904-48c2-811e-96240ee0fc25 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:22.352380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0632e-dd93-4928-89a9-a7aca813490b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:22.353890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0632e-dd93-4928-89a9-a7aca813490b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:22.538645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 92/190] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 01:05:22.540673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0632e-dd93-4928-89a9-a7aca813490b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:22.549262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3af40f-1904-48c2-811e-96240ee0fc25 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:22.594302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:22.617312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09050f44-87bf-4df2-abbe-b8a0ddccf434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:22.618430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09050f44-87bf-4df2-abbe-b8a0ddccf434 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:22.618993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09050f44-87bf-4df2-abbe-b8a0ddccf434 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:22.619821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09050f44-87bf-4df2-abbe-b8a0ddccf434 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:22.641518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09050f44-87bf-4df2-abbe-b8a0ddccf434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-09050f44-87bf-4df2-abbe-b8a0ddccf434 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:22.651947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:22.652925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:05:22.653718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:22.654361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 99/191] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:05:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:05:22.655685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09050f44-87bf-4df2-abbe-b8a0ddccf434 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:22.681634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0632e-dd93-4928-89a9-a7aca813490b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:22.698846 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:22.729651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a1a7e-2357-4975-b6bd-f478228a1a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:22.730592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a1a7e-2357-4975-b6bd-f478228a1a18 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:22.731026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a1a7e-2357-4975-b6bd-f478228a1a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:22.731456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a1a7e-2357-4975-b6bd-f478228a1a18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:22.732992 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a1a7e-2357-4975-b6bd-f478228a1a18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:22.734724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a1a7e-2357-4975-b6bd-f478228a1a18 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:22.736624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 93/192] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:05:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:22.742234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a1a7e-2357-4975-b6bd-f478228a1a18 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:22.842280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09050f44-87bf-4df2-abbe-b8a0ddccf434 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:22.897553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f332af-4745-4dc9-ab7d-ea0ce59ec704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:22.898768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f332af-4745-4dc9-ab7d-ea0ce59ec704 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:22.899241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f332af-4745-4dc9-ab7d-ea0ce59ec704 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:22.899987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f332af-4745-4dc9-ab7d-ea0ce59ec704 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:22.901655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f332af-4745-4dc9-ab7d-ea0ce59ec704 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:22.903352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f332af-4745-4dc9-ab7d-ea0ce59ec704 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:22.905209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 100/193] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:22 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:22.906139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f332af-4745-4dc9-ab7d-ea0ce59ec704 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:22.965541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a1a7e-2357-4975-b6bd-f478228a1a18 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:22.967801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:23.031841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aced33-e5ed-475b-9c5d-64792666ffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:23.033185 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aced33-e5ed-475b-9c5d-64792666ffeb None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:23.033642 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aced33-e5ed-475b-9c5d-64792666ffeb None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:23.035924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aced33-e5ed-475b-9c5d-64792666ffeb None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:23.036068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aced33-e5ed-475b-9c5d-64792666ffeb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:23.037093 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aced33-e5ed-475b-9c5d-64792666ffeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:23.037230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:23.039357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aced33-e5ed-475b-9c5d-64792666ffeb None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:23.061597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 94/194] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:05:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:23.061597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aced33-e5ed-475b-9c5d-64792666ffeb None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:23.067139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f332af-4745-4dc9-ab7d-ea0ce59ec704 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:23.133871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702736f-ab91-4293-9c18-807b36573f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:23.134920 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702736f-ab91-4293-9c18-807b36573f9d None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:23.135395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702736f-ab91-4293-9c18-807b36573f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:23.135915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702736f-ab91-4293-9c18-807b36573f9d None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:23.137173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702736f-ab91-4293-9c18-807b36573f9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:23.139977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702736f-ab91-4293-9c18-807b36573f9d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:23.215848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aced33-e5ed-475b-9c5d-64792666ffeb None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:23.262319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae48bf1-d52f-4d76-926b-b89bb9536b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:23.262937 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae48bf1-d52f-4d76-926b-b89bb9536b6b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:23.263216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae48bf1-d52f-4d76-926b-b89bb9536b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:23.263486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae48bf1-d52f-4d76-926b-b89bb9536b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:23.264262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae48bf1-d52f-4d76-926b-b89bb9536b6b None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=ac99f487657f41258cd0118165066bb1, user.id=5bd0770700d54763aa15987cb7f1a444) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:23.264574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae48bf1-d52f-4d76-926b-b89bb9536b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:23.266239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae48bf1-d52f-4d76-926b-b89bb9536b6b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:23.276000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 95/195] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:23 2022] GET /identity/v3/role_assignments?user.id=5bd0770700d54763aa15987cb7f1a444&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:23.276000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae48bf1-d52f-4d76-926b-b89bb9536b6b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:23.327195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:23.333752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 101/196] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:05:22 2022] POST /identity/v3/users => generated 297 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:05:23.453047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae48bf1-d52f-4d76-926b-b89bb9536b6b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:23.513293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4199d-9c1c-41d9-90ab-196fc7f5a063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:23.514111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4199d-9c1c-41d9-90ab-196fc7f5a063 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:23.514400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4199d-9c1c-41d9-90ab-196fc7f5a063 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:23.514677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4199d-9c1c-41d9-90ab-196fc7f5a063 None admin] PATH_INFO: `/v3/users/5bd0770700d54763aa15987cb7f1a444` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:23.516283 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4199d-9c1c-41d9-90ab-196fc7f5a063 None admin] RBAC: Authorizing `identity:get_user(user_id=5bd0770700d54763aa15987cb7f1a444)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:23.518067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4199d-9c1c-41d9-90ab-196fc7f5a063 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:23.521150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 96/197] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:23 2022] GET /identity/v3/users/5bd0770700d54763aa15987cb7f1a444 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:23.530185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702736f-ab91-4293-9c18-807b36573f9d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:23.700683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702736f-ab91-4293-9c18-807b36573f9d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:23.750814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea49744-0b17-4547-9123-25884af35b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:23.750814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea49744-0b17-4547-9123-25884af35b98 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:23.750814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea49744-0b17-4547-9123-25884af35b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:23.750814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea49744-0b17-4547-9123-25884af35b98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:23.763451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:23.836898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea49744-0b17-4547-9123-25884af35b98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:23.838465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea49744-0b17-4547-9123-25884af35b98 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:23.859025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273ccf0-a4f0-4fec-b9c4-3456791d55af None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:23.859534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273ccf0-a4f0-4fec-b9c4-3456791d55af None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:23.861729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273ccf0-a4f0-4fec-b9c4-3456791d55af None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:23.865137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 97/198] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:23.874820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768e735-4a34-4eb2-8521-18e82539d48c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:23.875343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768e735-4a34-4eb2-8521-18e82539d48c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:23.876082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b768e735-4a34-4eb2-8521-18e82539d48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:23.908516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b768e735-4a34-4eb2-8521-18e82539d48c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:23.910328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b768e735-4a34-4eb2-8521-18e82539d48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:24.013550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 102/199] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:24.023473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea49744-0b17-4547-9123-25884af35b98 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:24.037990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 98/200] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:24.117601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:24.215186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea49744-0b17-4547-9123-25884af35b98 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:24.276260 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95626c-f91f-42b1-9ee6-fbef2b2b7024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:24.277109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95626c-f91f-42b1-9ee6-fbef2b2b7024 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:24.277483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95626c-f91f-42b1-9ee6-fbef2b2b7024 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:24.277796 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95626c-f91f-42b1-9ee6-fbef2b2b7024 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:24.279204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95626c-f91f-42b1-9ee6-fbef2b2b7024 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:24.280931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c95626c-f91f-42b1-9ee6-fbef2b2b7024 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:24.283411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 103/201] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:24 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:24.291328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768e735-4a34-4eb2-8521-18e82539d48c None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:24.333992 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:24.473623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b768e735-4a34-4eb2-8521-18e82539d48c None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:24.513107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95626c-f91f-42b1-9ee6-fbef2b2b7024 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:24.555516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b307f-6f1c-4088-931d-b73f20a562c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:24.556178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b307f-6f1c-4088-931d-b73f20a562c4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:24.556458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b307f-6f1c-4088-931d-b73f20a562c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:24.556803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b307f-6f1c-4088-931d-b73f20a562c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:24.557569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b307f-6f1c-4088-931d-b73f20a562c4 None admin] RBAC: Adding query filter params (role.id=b9c95d84eaca4444b2c7b8b5eed61a59, scope.project.id=ac99f487657f41258cd0118165066bb1, user.id=5bd0770700d54763aa15987cb7f1a444) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:24.557941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b307f-6f1c-4088-931d-b73f20a562c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:24.560214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b307f-6f1c-4088-931d-b73f20a562c4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:24.569524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 99/202] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:24 2022] GET /identity/v3/role_assignments?user.id=5bd0770700d54763aa15987cb7f1a444&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=b9c95d84eaca4444b2c7b8b5eed61a59 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:24.775658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95626c-f91f-42b1-9ee6-fbef2b2b7024 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:24.805484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:24.851347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd2afa-75cb-4103-b995-1cffd806976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:24.852459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd2afa-75cb-4103-b995-1cffd806976e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:24.852931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd2afa-75cb-4103-b995-1cffd806976e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:24.853406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd2afa-75cb-4103-b995-1cffd806976e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:24.877970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dd2afa-75cb-4103-b995-1cffd806976e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-c2dd2afa-75cb-4103-b995-1cffd806976e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:24.884155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:24.885162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:24.886550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 104/203] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:24.894257 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b307f-6f1c-4088-931d-b73f20a562c4 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:25.108420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:25.126661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b307f-6f1c-4088-931d-b73f20a562c4 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:25.204003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf105195-1ee4-4eff-85d5-4a8385fdc1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:25.204639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf105195-1ee4-4eff-85d5-4a8385fdc1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:25.204983 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf105195-1ee4-4eff-85d5-4a8385fdc1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:25.205286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf105195-1ee4-4eff-85d5-4a8385fdc1c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:25.206217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf105195-1ee4-4eff-85d5-4a8385fdc1c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:25.206584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf105195-1ee4-4eff-85d5-4a8385fdc1c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:25.208194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf105195-1ee4-4eff-85d5-4a8385fdc1c4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:25.219198 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 100/204] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:25.227450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd2afa-75cb-4103-b995-1cffd806976e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:25.408093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd2afa-75cb-4103-b995-1cffd806976e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:25.463709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42172b-84fc-4b3b-a24c-946d721111c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:25.464397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42172b-84fc-4b3b-a24c-946d721111c5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:25.464685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42172b-84fc-4b3b-a24c-946d721111c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:25.464989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42172b-84fc-4b3b-a24c-946d721111c5 None admin] PATH_INFO: `/v3/users/fc34bd37fe314414a940e403c375234c` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:25.466319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42172b-84fc-4b3b-a24c-946d721111c5 None admin] RBAC: Authorizing `identity:get_user(user_id=fc34bd37fe314414a940e403c375234c)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:25.467987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42172b-84fc-4b3b-a24c-946d721111c5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:25.469801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 105/205] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:25 2022] GET /identity/v3/users/fc34bd37fe314414a940e403c375234c => generated 289 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:25.476715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf105195-1ee4-4eff-85d5-4a8385fdc1c4 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:25.697595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf105195-1ee4-4eff-85d5-4a8385fdc1c4 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:25.770089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748a9cf-d307-4065-bd4c-7d46a88fe1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:25.771112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748a9cf-d307-4065-bd4c-7d46a88fe1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:25.771613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748a9cf-d307-4065-bd4c-7d46a88fe1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:25.772100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748a9cf-d307-4065-bd4c-7d46a88fe1d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:25.856483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:25.936950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748a9cf-d307-4065-bd4c-7d46a88fe1d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:25.940086 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748a9cf-d307-4065-bd4c-7d46a88fe1d4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:26.148636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 101/206] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:26.156081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42172b-84fc-4b3b-a24c-946d721111c5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:26.268159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:26.401652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42172b-84fc-4b3b-a24c-946d721111c5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:26.492957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e53a01-a96a-4ac0-8b15-769d61187975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:26.494125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e53a01-a96a-4ac0-8b15-769d61187975 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:26.494631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e53a01-a96a-4ac0-8b15-769d61187975 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:26.495128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e53a01-a96a-4ac0-8b15-769d61187975 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:26.501549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e53a01-a96a-4ac0-8b15-769d61187975 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:26.505171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e53a01-a96a-4ac0-8b15-769d61187975 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:26.508868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 106/207] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:26 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:26.533965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748a9cf-d307-4065-bd4c-7d46a88fe1d4 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:26.700836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:26.891117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748a9cf-d307-4065-bd4c-7d46a88fe1d4 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:27.013374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c2980f-6175-489b-b46b-36c90f67c52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:27.014859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2980f-6175-489b-b46b-36c90f67c52b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:27.020088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2980f-6175-489b-b46b-36c90f67c52b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:27.021011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c2980f-6175-489b-b46b-36c90f67c52b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:27.022779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2980f-6175-489b-b46b-36c90f67c52b None admin] RBAC: Adding query filter params (role.id=558d573b58fa4ba59d56e645fd5a0000, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005, user.id=fc34bd37fe314414a940e403c375234c) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:27.023656 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2980f-6175-489b-b46b-36c90f67c52b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:27.026729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c2980f-6175-489b-b46b-36c90f67c52b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:27.047800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 102/208] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:26 2022] GET /identity/v3/role_assignments?user.id=fc34bd37fe314414a940e403c375234c&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=558d573b58fa4ba59d56e645fd5a0000 => generated 608 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:27.484912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb0087-f6c5-47fb-a499-960187d33f8c None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:27.486184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb0087-f6c5-47fb-a499-960187d33f8c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:27.486781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb0087-f6c5-47fb-a499-960187d33f8c None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:27.489129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 107/209] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:27.499423 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f8a54-c514-4fd2-b8f2-89eb6ae911e1 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:27.501018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f8a54-c514-4fd2-b8f2-89eb6ae911e1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:27.501847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f8a54-c514-4fd2-b8f2-89eb6ae911e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:27.538747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-389f8a54-c514-4fd2-b8f2-89eb6ae911e1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:27.541323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-389f8a54-c514-4fd2-b8f2-89eb6ae911e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:27.643615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d0a4-13d3-4dbe-adfc-8b9c40f716fc None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:27.645147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d0a4-13d3-4dbe-adfc-8b9c40f716fc None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:27.645908 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b44d0a4-13d3-4dbe-adfc-8b9c40f716fc None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:27.649844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 108/210] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:05:27.667271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b0f36-2f04-42bc-9ca9-b1c0c59a198b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:27.668322 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b0f36-2f04-42bc-9ca9-b1c0c59a198b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:27.669106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b0f36-2f04-42bc-9ca9-b1c0c59a198b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:27.703578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5c1b0f36-2f04-42bc-9ca9-b1c0c59a198b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:27.706548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5c1b0f36-2f04-42bc-9ca9-b1c0c59a198b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:27.712602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 103/211] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:27.848500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 109/212] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:28.136378 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f8a54-c514-4fd2-b8f2-89eb6ae911e1 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:28.201131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:28.273123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8a821-cba8-4bf5-b8a8-722cb52f1c75 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:28.273992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8a821-cba8-4bf5-b8a8-722cb52f1c75 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:28.274684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa8a821-cba8-4bf5-b8a8-722cb52f1c75 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:28.277435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 110/213] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:05:28.286865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374b3ab-7f4b-431f-b3c7-8586e12d0c6b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:28.287801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374b3ab-7f4b-431f-b3c7-8586e12d0c6b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:28.288575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374b3ab-7f4b-431f-b3c7-8586e12d0c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:28.301572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f8a54-c514-4fd2-b8f2-89eb6ae911e1 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:28.320364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5374b3ab-7f4b-431f-b3c7-8586e12d0c6b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:28.322926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5374b3ab-7f4b-431f-b3c7-8586e12d0c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:28.346466 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65e4d1-69fe-4617-96bc-19ebfe8ec583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:28.348247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65e4d1-69fe-4617-96bc-19ebfe8ec583 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:28.348718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65e4d1-69fe-4617-96bc-19ebfe8ec583 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:28.349244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65e4d1-69fe-4617-96bc-19ebfe8ec583 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:28.370139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65e4d1-69fe-4617-96bc-19ebfe8ec583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-3f65e4d1-69fe-4617-96bc-19ebfe8ec583 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:28.376577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:28.377819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:28.378704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:28.387993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 104/214] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:05:28.387993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65e4d1-69fe-4617-96bc-19ebfe8ec583 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:28.391671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:28.444769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 111/215] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:28.446781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374b3ab-7f4b-431f-b3c7-8586e12d0c6b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:28.576349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65e4d1-69fe-4617-96bc-19ebfe8ec583 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:28.591456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:28.644070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e41af1-8fb6-4ccb-876f-303899159d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:28.644774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e41af1-8fb6-4ccb-876f-303899159d25 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:28.645260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e41af1-8fb6-4ccb-876f-303899159d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:28.645534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e41af1-8fb6-4ccb-876f-303899159d25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:28.658147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374b3ab-7f4b-431f-b3c7-8586e12d0c6b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:28.666127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e41af1-8fb6-4ccb-876f-303899159d25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-92e41af1-8fb6-4ccb-876f-303899159d25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:28.674874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:28.677177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 105/216] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:05:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:28.678890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88468b-c321-49e3-8d09-c19f8fbab556 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:28.680393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88468b-c321-49e3-8d09-c19f8fbab556 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:28.680393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88468b-c321-49e3-8d09-c19f8fbab556 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:28.682418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 106/217] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:28.689882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88468b-c321-49e3-8d09-c19f8fbab556 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:28.736558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146821a6-5acf-4039-a23b-4dfd806ec29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:28.737556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146821a6-5acf-4039-a23b-4dfd806ec29a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:28.738537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146821a6-5acf-4039-a23b-4dfd806ec29a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:28.738992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146821a6-5acf-4039-a23b-4dfd806ec29a None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:28.740053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146821a6-5acf-4039-a23b-4dfd806ec29a None admin] RBAC: Adding query filter params (name=member) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:28.740559 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146821a6-5acf-4039-a23b-4dfd806ec29a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:28.742993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146821a6-5acf-4039-a23b-4dfd806ec29a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:28.760930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 112/218] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:28.763184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4d5d7-76b3-4558-9848-ca3f849ad166 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:28.763677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4d5d7-76b3-4558-9848-ca3f849ad166 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:28.764116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4d5d7-76b3-4558-9848-ca3f849ad166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:28.793294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-1dd4d5d7-76b3-4558-9848-ca3f849ad166 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:28.795309 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1dd4d5d7-76b3-4558-9848-ca3f849ad166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:28.848567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:28.890126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88468b-c321-49e3-8d09-c19f8fbab556 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:28.905359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 113/219] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:28.906291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4d5d7-76b3-4558-9848-ca3f849ad166 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:28.955602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd5235-fff7-4acd-a2ea-172c2cf85a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:28.957660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd5235-fff7-4acd-a2ea-172c2cf85a3c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:28.958447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd5235-fff7-4acd-a2ea-172c2cf85a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:28.959217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dd5235-fff7-4acd-a2ea-172c2cf85a3c None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:28.960824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd5235-fff7-4acd-a2ea-172c2cf85a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:28.961644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd5235-fff7-4acd-a2ea-172c2cf85a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:28.964692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dd5235-fff7-4acd-a2ea-172c2cf85a3c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:28.981401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 107/220] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:28.983432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd5235-fff7-4acd-a2ea-172c2cf85a3c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:29.047210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4d5d7-76b3-4558-9848-ca3f849ad166 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:29.048630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:29.109541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e47cd7-9243-4a14-a867-3f3f6c587392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:29.110672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e47cd7-9243-4a14-a867-3f3f6c587392 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:29.111168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e47cd7-9243-4a14-a867-3f3f6c587392 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:29.111694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e47cd7-9243-4a14-a867-3f3f6c587392 None admin] PATH_INFO: `/v3/users/96e2c26992bc4dc48f6e11cabb12c284` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:29.134926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e47cd7-9243-4a14-a867-3f3f6c587392 None admin] RBAC: Authorizing `identity:get_user(user_id=96e2c26992bc4dc48f6e11cabb12c284)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:29.136598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e47cd7-9243-4a14-a867-3f3f6c587392 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:29.138430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 114/221] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:28 2022] GET /identity/v3/users/96e2c26992bc4dc48f6e11cabb12c284 => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:29.144713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e47cd7-9243-4a14-a867-3f3f6c587392 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:29.171706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:29.204852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dd5235-fff7-4acd-a2ea-172c2cf85a3c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:29.265641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f9da6-0c0d-41e5-a871-c6b2598b878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:29.266386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f9da6-0c0d-41e5-a871-c6b2598b878e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:29.266720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f9da6-0c0d-41e5-a871-c6b2598b878e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:29.267136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f9da6-0c0d-41e5-a871-c6b2598b878e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:29.268520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f9da6-0c0d-41e5-a871-c6b2598b878e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:29.270744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f9da6-0c0d-41e5-a871-c6b2598b878e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:29.275622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 108/222] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:05:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:29.275622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f9da6-0c0d-41e5-a871-c6b2598b878e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:29.278212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e47cd7-9243-4a14-a867-3f3f6c587392 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:29.326763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d6c9f-fcb5-4c2f-9b2e-33f961bd1109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:29.327895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d6c9f-fcb5-4c2f-9b2e-33f961bd1109 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:29.328369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d6c9f-fcb5-4c2f-9b2e-33f961bd1109 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:29.328856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d6c9f-fcb5-4c2f-9b2e-33f961bd1109 None admin] PATH_INFO: `/v3/users/5bd0770700d54763aa15987cb7f1a444` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:29.330904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d6c9f-fcb5-4c2f-9b2e-33f961bd1109 None admin] RBAC: Authorizing `identity:get_user(user_id=5bd0770700d54763aa15987cb7f1a444)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:29.334159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d6c9f-fcb5-4c2f-9b2e-33f961bd1109 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:29.337320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 115/223] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:29 2022] GET /identity/v3/users/5bd0770700d54763aa15987cb7f1a444 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:29.338936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d6c9f-fcb5-4c2f-9b2e-33f961bd1109 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:29.402658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:29.412723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f9da6-0c0d-41e5-a871-c6b2598b878e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:29.457810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a215022-8548-483d-8efb-303318a3d235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:29.458689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215022-8548-483d-8efb-303318a3d235 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:29.458981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215022-8548-483d-8efb-303318a3d235 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:29.459256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a215022-8548-483d-8efb-303318a3d235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:29.489752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:29.513805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d6c9f-fcb5-4c2f-9b2e-33f961bd1109 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:29.542140 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a215022-8548-483d-8efb-303318a3d235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:29.544851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a215022-8548-483d-8efb-303318a3d235 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:29.578109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b21c4-f02c-4dcf-ac3c-5e361a139e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:29.579038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b21c4-f02c-4dcf-ac3c-5e361a139e5e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:29.579447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b21c4-f02c-4dcf-ac3c-5e361a139e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:29.579891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b21c4-f02c-4dcf-ac3c-5e361a139e5e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:29.593730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b21c4-f02c-4dcf-ac3c-5e361a139e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-fd7b21c4-f02c-4dcf-ac3c-5e361a139e5e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:29.599998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:29.601654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 116/224] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:05:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:29.603098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b21c4-f02c-4dcf-ac3c-5e361a139e5e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:29.682548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:29.691076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 109/225] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:29.692495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a215022-8548-483d-8efb-303318a3d235 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:29.755270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b21c4-f02c-4dcf-ac3c-5e361a139e5e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:29.802480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cccf4-c1f9-4240-b225-ca36230bb7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:29.803195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cccf4-c1f9-4240-b225-ca36230bb7a1 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:29.803626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cccf4-c1f9-4240-b225-ca36230bb7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:29.804022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529cccf4-c1f9-4240-b225-ca36230bb7a1 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:29.804782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cccf4-c1f9-4240-b225-ca36230bb7a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:29.806387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529cccf4-c1f9-4240-b225-ca36230bb7a1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:29.842487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a215022-8548-483d-8efb-303318a3d235 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:29.891900 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b13bb-a088-42ef-abcf-59002ee5e5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:29.892729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b13bb-a088-42ef-abcf-59002ee5e5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:29.893067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b13bb-a088-42ef-abcf-59002ee5e5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:29.893391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b13bb-a088-42ef-abcf-59002ee5e5a5 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:29.896892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b13bb-a088-42ef-abcf-59002ee5e5a5 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:29.898854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b13bb-a088-42ef-abcf-59002ee5e5a5 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:29.901226 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 110/226] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:29 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:29.902633 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b13bb-a088-42ef-abcf-59002ee5e5a5 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:29.966647 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 117/227] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:05:29 2022] POST /identity/v3/users => generated 297 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:05:29.966647 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cccf4-c1f9-4240-b225-ca36230bb7a1 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:29.973849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:29.994200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:30.061005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b13bb-a088-42ef-abcf-59002ee5e5a5 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:30.117357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb250f-bbe0-4a7a-bd18-ef1d3cefe730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:30.118168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb250f-bbe0-4a7a-bd18-ef1d3cefe730 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:30.118639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb250f-bbe0-4a7a-bd18-ef1d3cefe730 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:30.119205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb250f-bbe0-4a7a-bd18-ef1d3cefe730 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:30.120965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb250f-bbe0-4a7a-bd18-ef1d3cefe730 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:30.120965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb250f-bbe0-4a7a-bd18-ef1d3cefe730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:30.124919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb250f-bbe0-4a7a-bd18-ef1d3cefe730 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:30.133590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 111/228] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:05:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:30.134377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb250f-bbe0-4a7a-bd18-ef1d3cefe730 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:30.167450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529cccf4-c1f9-4240-b225-ca36230bb7a1 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:30.233505 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07280c7-5848-45a3-801c-2cad60ce1368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:30.234516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07280c7-5848-45a3-801c-2cad60ce1368 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:30.234939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07280c7-5848-45a3-801c-2cad60ce1368 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:30.235342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07280c7-5848-45a3-801c-2cad60ce1368 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:30.237099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07280c7-5848-45a3-801c-2cad60ce1368 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:30.239974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07280c7-5848-45a3-801c-2cad60ce1368 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:30.242801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 118/229] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:29 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:30.243835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07280c7-5848-45a3-801c-2cad60ce1368 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:30.298751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb250f-bbe0-4a7a-bd18-ef1d3cefe730 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:30.334420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:30.352497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878cd1-bc15-411c-bef4-c33a32db51db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:30.353210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878cd1-bc15-411c-bef4-c33a32db51db None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:30.354666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878cd1-bc15-411c-bef4-c33a32db51db None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:30.354666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d878cd1-bc15-411c-bef4-c33a32db51db None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1/users/5bd0770700d54763aa15987cb7f1a444/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:30.357200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d878cd1-bc15-411c-bef4-c33a32db51db None admin] RBAC: Authorizing `identity:create_grant(project_id=ac99f487657f41258cd0118165066bb1, user_id=5bd0770700d54763aa15987cb7f1a444, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:30.359252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d878cd1-bc15-411c-bef4-c33a32db51db None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:30.370584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:30.415334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07280c7-5848-45a3-801c-2cad60ce1368 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:30.456524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 112/230] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:05:30 2022] PUT /identity/v3/projects/ac99f487657f41258cd0118165066bb1/users/5bd0770700d54763aa15987cb7f1a444/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:05:30.461188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878cd1-bc15-411c-bef4-c33a32db51db None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:30.487112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6cbff-a636-4906-86af-d1c01bcc9349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:30.487843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6cbff-a636-4906-86af-d1c01bcc9349 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:30.488147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6cbff-a636-4906-86af-d1c01bcc9349 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:30.488436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d6cbff-a636-4906-86af-d1c01bcc9349 None admin] PATH_INFO: `/v3/users/5bd0770700d54763aa15987cb7f1a444` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:30.489819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6cbff-a636-4906-86af-d1c01bcc9349 None admin] RBAC: Authorizing `identity:get_user(user_id=5bd0770700d54763aa15987cb7f1a444)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:30.491521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d6cbff-a636-4906-86af-d1c01bcc9349 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:30.493514 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 119/231] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:30 2022] GET /identity/v3/users/5bd0770700d54763aa15987cb7f1a444 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:30.499934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6cbff-a636-4906-86af-d1c01bcc9349 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:30.664801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d6cbff-a636-4906-86af-d1c01bcc9349 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:30.711293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d878cd1-bc15-411c-bef4-c33a32db51db None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:30.728646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038ac61-02a6-47d9-bd79-161e9cc734cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:30.730190 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038ac61-02a6-47d9-bd79-161e9cc734cf None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:30.731094 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038ac61-02a6-47d9-bd79-161e9cc734cf None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:30.731965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038ac61-02a6-47d9-bd79-161e9cc734cf None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:30.734267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038ac61-02a6-47d9-bd79-161e9cc734cf None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:30.737359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038ac61-02a6-47d9-bd79-161e9cc734cf None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:30.740680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 120/232] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:30 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:30.747539 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038ac61-02a6-47d9-bd79-161e9cc734cf None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:30.770976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:30.802836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4939613-803f-41cf-a135-4edfe4155742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:30.804003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4939613-803f-41cf-a135-4edfe4155742 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:30.804486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4939613-803f-41cf-a135-4edfe4155742 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:30.804950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4939613-803f-41cf-a135-4edfe4155742 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:30.806295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4939613-803f-41cf-a135-4edfe4155742 None admin] RBAC: Adding query filter params (role.id=e5a65ae2d0884e419cdbc70c9223eac2, scope.project.id=ac99f487657f41258cd0118165066bb1, user.id=96e2c26992bc4dc48f6e11cabb12c284) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:30.806837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4939613-803f-41cf-a135-4edfe4155742 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:30.811267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4939613-803f-41cf-a135-4edfe4155742 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:30.832739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 113/233] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:30 2022] GET /identity/v3/role_assignments?user.id=96e2c26992bc4dc48f6e11cabb12c284&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 259 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:30.982826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038ac61-02a6-47d9-bd79-161e9cc734cf None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:31.063333 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aeb7b-d53c-46d8-8d6e-f7008f9d1495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:31.064071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aeb7b-d53c-46d8-8d6e-f7008f9d1495 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:31.064450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aeb7b-d53c-46d8-8d6e-f7008f9d1495 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:31.064747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aeb7b-d53c-46d8-8d6e-f7008f9d1495 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1/users/5bd0770700d54763aa15987cb7f1a444/roles/b9c95d84eaca4444b2c7b8b5eed61a59` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:31.067523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aeb7b-d53c-46d8-8d6e-f7008f9d1495 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac99f487657f41258cd0118165066bb1, user_id=5bd0770700d54763aa15987cb7f1a444, role_id=b9c95d84eaca4444b2c7b8b5eed61a59)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:31.069488 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aeb7b-d53c-46d8-8d6e-f7008f9d1495 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:31.206878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 121/234] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:05:30 2022] PUT /identity/v3/projects/ac99f487657f41258cd0118165066bb1/users/5bd0770700d54763aa15987cb7f1a444/roles/b9c95d84eaca4444b2c7b8b5eed61a59 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:05:34.625606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e64de-3a0c-4e1a-a033-49e45aa8f433 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:34.627223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e64de-3a0c-4e1a-a033-49e45aa8f433 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:34.628006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e64de-3a0c-4e1a-a033-49e45aa8f433 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:34.631679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 114/235] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:34.641557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4d382-a04d-4c74-83c3-14555821f9b9 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:34.643005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4d382-a04d-4c74-83c3-14555821f9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:34.643805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4d382-a04d-4c74-83c3-14555821f9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:34.684319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5cf4d382-a04d-4c74-83c3-14555821f9b9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:34.686854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5cf4d382-a04d-4c74-83c3-14555821f9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:34.745967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b032cc-e381-41f0-ae5e-cb31f5f17c3f None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:34.746970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b032cc-e381-41f0-ae5e-cb31f5f17c3f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:34.747949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b032cc-e381-41f0-ae5e-cb31f5f17c3f None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:34.751008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 115/236] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:05:34.761884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0e991-1c7f-46ba-a97c-5f2965201c5a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:34.763086 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0e991-1c7f-46ba-a97c-5f2965201c5a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:34.763947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0e991-1c7f-46ba-a97c-5f2965201c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:34.802065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-2cb0e991-1c7f-46ba-a97c-5f2965201c5a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:34.804980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2cb0e991-1c7f-46ba-a97c-5f2965201c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:34.865452 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:34.903773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 122/237] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:34.992391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:35.048520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 116/238] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:35.085607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7815d7-ced2-4e37-a68d-f83f5fc21f0b None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:35.086024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7815d7-ced2-4e37-a68d-f83f5fc21f0b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:35.086657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7815d7-ced2-4e37-a68d-f83f5fc21f0b None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:35.089282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 123/239] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:35.099534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6493d72-7f22-4758-ad08-9cfc7dd4dc7e None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:35.100431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6493d72-7f22-4758-ad08-9cfc7dd4dc7e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:35.101274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6493d72-7f22-4758-ad08-9cfc7dd4dc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:35.138578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-c6493d72-7f22-4758-ad08-9cfc7dd4dc7e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:35.141113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c6493d72-7f22-4758-ad08-9cfc7dd4dc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:35.309134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb65b8-098f-48c1-acc3-c54be8d8ebbe None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:35.310097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb65b8-098f-48c1-acc3-c54be8d8ebbe None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:35.310624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 117/240] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:35.311185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb65b8-098f-48c1-acc3-c54be8d8ebbe None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:35.316715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 124/241] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:35.339354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22566da-aae2-4a89-849d-8d43521de80d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:35.340070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22566da-aae2-4a89-849d-8d43521de80d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:35.340562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22566da-aae2-4a89-849d-8d43521de80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:35.385372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d22566da-aae2-4a89-849d-8d43521de80d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:35.387381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d22566da-aae2-4a89-849d-8d43521de80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:35.542231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 118/242] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:35.563522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb65b8-098f-48c1-acc3-c54be8d8ebbe None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:35.692837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22566da-aae2-4a89-849d-8d43521de80d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:35.792766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb65b8-098f-48c1-acc3-c54be8d8ebbe None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:35.797734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:35.886871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c55a271-6647-4178-bb30-4f46c694e2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:35.888325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55a271-6647-4178-bb30-4f46c694e2be None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:35.889391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55a271-6647-4178-bb30-4f46c694e2be None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:35.890068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c55a271-6647-4178-bb30-4f46c694e2be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:35.892391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55a271-6647-4178-bb30-4f46c694e2be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:35.897033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c55a271-6647-4178-bb30-4f46c694e2be None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:35.903866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 125/243] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:05:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:35.909876 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7aab1-a96f-45d1-988a-250930dec9bb None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:35.911226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7aab1-a96f-45d1-988a-250930dec9bb None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:35.911940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7aab1-a96f-45d1-988a-250930dec9bb None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:35.916346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 126/244] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:35 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:35.917430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7aab1-a96f-45d1-988a-250930dec9bb None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:35.920675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22566da-aae2-4a89-849d-8d43521de80d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:35.993461 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861e56a-a4cb-4d62-8c6b-daf180d0b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:35.994365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861e56a-a4cb-4d62-8c6b-daf180d0b004 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:35.994831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861e56a-a4cb-4d62-8c6b-daf180d0b004 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:35.995703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861e56a-a4cb-4d62-8c6b-daf180d0b004 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:36.008256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:36.010875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861e56a-a4cb-4d62-8c6b-daf180d0b004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-a861e56a-a4cb-4d62-8c6b-daf180d0b004 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:36.017013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:36.025321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 119/245] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:36.026825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861e56a-a4cb-4d62-8c6b-daf180d0b004 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:36.058959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7aab1-a96f-45d1-988a-250930dec9bb None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:36.111833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcba18-a15d-4252-a5a1-c40ff30edea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:36.112732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcba18-a15d-4252-a5a1-c40ff30edea6 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:36.113692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcba18-a15d-4252-a5a1-c40ff30edea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:36.114150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcba18-a15d-4252-a5a1-c40ff30edea6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:36.131497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbcba18-a15d-4252-a5a1-c40ff30edea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-9fbcba18-a15d-4252-a5a1-c40ff30edea6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:36.140161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:36.141160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:36.142060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 127/246] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:05:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:36.144062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9635f-32ed-433d-bbd1-f87e6fd2309b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:36.144547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9635f-32ed-433d-bbd1-f87e6fd2309b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:36.145027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9635f-32ed-433d-bbd1-f87e6fd2309b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:36.168372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-98a9635f-32ed-433d-bbd1-f87e6fd2309b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:36.169661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-98a9635f-32ed-433d-bbd1-f87e6fd2309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:36.170717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:36.174290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861e56a-a4cb-4d62-8c6b-daf180d0b004 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:36.229928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90015b92-2130-487c-8eb4-c97238b58379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:36.230621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90015b92-2130-487c-8eb4-c97238b58379 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:36.230900 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90015b92-2130-487c-8eb4-c97238b58379 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:36.231173 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90015b92-2130-487c-8eb4-c97238b58379 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:36.231886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90015b92-2130-487c-8eb4-c97238b58379 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:36.233429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90015b92-2130-487c-8eb4-c97238b58379 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:36.280195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 128/247] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:36.281784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9635f-32ed-433d-bbd1-f87e6fd2309b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:36.346232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 120/248] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:05:36 2022] POST /identity/v3/users => generated 293 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:05:36.347836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90015b92-2130-487c-8eb4-c97238b58379 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:36.400130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:36.477840 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9635f-32ed-433d-bbd1-f87e6fd2309b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:36.557967 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453c605-451d-422c-a7e0-2d50e61f3a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:36.559153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453c605-451d-422c-a7e0-2d50e61f3a35 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:36.559760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453c605-451d-422c-a7e0-2d50e61f3a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:36.560328 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453c605-451d-422c-a7e0-2d50e61f3a35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:36.562052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90015b92-2130-487c-8eb4-c97238b58379 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:36.583968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453c605-451d-422c-a7e0-2d50e61f3a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-0453c605-451d-422c-a7e0-2d50e61f3a35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:36.593841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:36.595324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 129/249] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:36.599858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453c605-451d-422c-a7e0-2d50e61f3a35 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:36.639505 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75977e9f-6c6a-4126-b7c3-b355ef7f7270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:36.640297 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75977e9f-6c6a-4126-b7c3-b355ef7f7270 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:36.640572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75977e9f-6c6a-4126-b7c3-b355ef7f7270 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:36.640932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75977e9f-6c6a-4126-b7c3-b355ef7f7270 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:36.641909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75977e9f-6c6a-4126-b7c3-b355ef7f7270 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:36.642210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75977e9f-6c6a-4126-b7c3-b355ef7f7270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:36.644324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75977e9f-6c6a-4126-b7c3-b355ef7f7270 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:36.662166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 121/250] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:36.667586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75977e9f-6c6a-4126-b7c3-b355ef7f7270 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:36.697533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:36.812542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453c605-451d-422c-a7e0-2d50e61f3a35 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:36.838656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:36.870708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9204025-99df-44f7-b169-3be0b76ba025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:36.871755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9204025-99df-44f7-b169-3be0b76ba025 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:36.872041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9204025-99df-44f7-b169-3be0b76ba025 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:36.872316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9204025-99df-44f7-b169-3be0b76ba025 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:36.873129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9204025-99df-44f7-b169-3be0b76ba025 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:36.873436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9204025-99df-44f7-b169-3be0b76ba025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:36.875806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9204025-99df-44f7-b169-3be0b76ba025 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:36.887815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 130/251] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:36.888708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9204025-99df-44f7-b169-3be0b76ba025 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:36.941063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75977e9f-6c6a-4126-b7c3-b355ef7f7270 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:37.014703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf657b0a-bae5-4459-8fc3-08dff4636f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:37.017328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf657b0a-bae5-4459-8fc3-08dff4636f38 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:37.018151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf657b0a-bae5-4459-8fc3-08dff4636f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:37.018864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf657b0a-bae5-4459-8fc3-08dff4636f38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:37.046211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf657b0a-bae5-4459-8fc3-08dff4636f38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-cf657b0a-bae5-4459-8fc3-08dff4636f38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:37.063715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:37.065105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:05:37.066430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:37.071001 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 122/252] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:05:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:37.072786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf657b0a-bae5-4459-8fc3-08dff4636f38 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:37.073453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9204025-99df-44f7-b169-3be0b76ba025 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:37.140507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2e902-1f04-4a63-96bd-698e7213e1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:37.141792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2e902-1f04-4a63-96bd-698e7213e1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:37.142383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2e902-1f04-4a63-96bd-698e7213e1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:37.143033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2e902-1f04-4a63-96bd-698e7213e1d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:37.144992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2e902-1f04-4a63-96bd-698e7213e1d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:37.145747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2e902-1f04-4a63-96bd-698e7213e1d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:37.148511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2e902-1f04-4a63-96bd-698e7213e1d3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:37.166474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 131/253] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:37.176847 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2e902-1f04-4a63-96bd-698e7213e1d3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:37.251455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:37.268102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:37.320296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf657b0a-bae5-4459-8fc3-08dff4636f38 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:37.388714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199192c-0dfd-470a-a1a2-3689d9ae3d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:37.389928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199192c-0dfd-470a-a1a2-3689d9ae3d69 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:37.390517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199192c-0dfd-470a-a1a2-3689d9ae3d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:37.391116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199192c-0dfd-470a-a1a2-3689d9ae3d69 None admin] PATH_INFO: `/v3/users/188b456e7b414043a219b9e59266cbf2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:37.393388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2e902-1f04-4a63-96bd-698e7213e1d3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:37.418071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199192c-0dfd-470a-a1a2-3689d9ae3d69 None admin] RBAC: Authorizing `identity:get_user(user_id=188b456e7b414043a219b9e59266cbf2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:37.420953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199192c-0dfd-470a-a1a2-3689d9ae3d69 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:37.424375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 123/254] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:37 2022] GET /identity/v3/users/188b456e7b414043a219b9e59266cbf2 => generated 297 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:37.433341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199192c-0dfd-470a-a1a2-3689d9ae3d69 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:37.473132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9734c-c4e4-4c76-85c3-1b25310893c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:37.475254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9734c-c4e4-4c76-85c3-1b25310893c7 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:37.475952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9734c-c4e4-4c76-85c3-1b25310893c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:37.476586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9734c-c4e4-4c76-85c3-1b25310893c7 None admin] PATH_INFO: `/v3/users/5bd0770700d54763aa15987cb7f1a444` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:37.479256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9734c-c4e4-4c76-85c3-1b25310893c7 None admin] RBAC: Authorizing `identity:get_user(user_id=5bd0770700d54763aa15987cb7f1a444)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:37.483473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9734c-c4e4-4c76-85c3-1b25310893c7 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:37.488765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 132/255] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:37 2022] GET /identity/v3/users/5bd0770700d54763aa15987cb7f1a444 => generated 283 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:37.490525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9734c-c4e4-4c76-85c3-1b25310893c7 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:37.644932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199192c-0dfd-470a-a1a2-3689d9ae3d69 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:37.696438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9734c-c4e4-4c76-85c3-1b25310893c7 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:37.710855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:37.727077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2de84-ba6e-410f-aae4-a3192cd9dd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:37.728969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2de84-ba6e-410f-aae4-a3192cd9dd78 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:37.729968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2de84-ba6e-410f-aae4-a3192cd9dd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:37.730616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2de84-ba6e-410f-aae4-a3192cd9dd78 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:37.732536 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2de84-ba6e-410f-aae4-a3192cd9dd78 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:37.733253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2de84-ba6e-410f-aae4-a3192cd9dd78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:37.736119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2de84-ba6e-410f-aae4-a3192cd9dd78 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:37.741731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:37.758377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 124/256] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:05:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:37.760867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2de84-ba6e-410f-aae4-a3192cd9dd78 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:37.765699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b1aef-10a8-4591-80af-e7767be8bffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:37.766946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b1aef-10a8-4591-80af-e7767be8bffd None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:37.767468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b1aef-10a8-4591-80af-e7767be8bffd None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:37.768070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b1aef-10a8-4591-80af-e7767be8bffd None admin] PATH_INFO: `/v3/users/96e2c26992bc4dc48f6e11cabb12c284` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:37.771636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b1aef-10a8-4591-80af-e7767be8bffd None admin] RBAC: Authorizing `identity:get_user(user_id=96e2c26992bc4dc48f6e11cabb12c284)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:37.774546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b1aef-10a8-4591-80af-e7767be8bffd None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:37.777730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 133/257] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:37 2022] GET /identity/v3/users/96e2c26992bc4dc48f6e11cabb12c284 => generated 297 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:37.784150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b1aef-10a8-4591-80af-e7767be8bffd None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:37.977783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2de84-ba6e-410f-aae4-a3192cd9dd78 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:38.026443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b1aef-10a8-4591-80af-e7767be8bffd None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:38.045792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e2765-ab0f-403b-94dd-f49bde82a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:38.047106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e2765-ab0f-403b-94dd-f49bde82a188 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:38.047854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e2765-ab0f-403b-94dd-f49bde82a188 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:38.048443 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e2765-ab0f-403b-94dd-f49bde82a188 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:38.078203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c57e1-3967-40f1-a0d8-2ccaf3e33cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:38.078920 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c57e1-3967-40f1-a0d8-2ccaf3e33cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:38.079301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c57e1-3967-40f1-a0d8-2ccaf3e33cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:38.079705 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c57e1-3967-40f1-a0d8-2ccaf3e33cf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:38.150922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:38.187195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:38.196935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c57e1-3967-40f1-a0d8-2ccaf3e33cf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:38.197868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e2765-ab0f-403b-94dd-f49bde82a188 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:38.199026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c57e1-3967-40f1-a0d8-2ccaf3e33cf4 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:38.200868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e2765-ab0f-403b-94dd-f49bde82a188 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:38.356181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 134/258] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:38.358001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c57e1-3967-40f1-a0d8-2ccaf3e33cf4 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:38.436202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 125/259] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:38.438336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e2765-ab0f-403b-94dd-f49bde82a188 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:38.533967 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:38.575101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c57e1-3967-40f1-a0d8-2ccaf3e33cf4 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:38.624592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852a636-01da-44f3-b4ca-9c59d31fedb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:38.626287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852a636-01da-44f3-b4ca-9c59d31fedb0 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:38.626723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852a636-01da-44f3-b4ca-9c59d31fedb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:38.627233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852a636-01da-44f3-b4ca-9c59d31fedb0 None admin] PATH_INFO: `/v3/users/5bd0770700d54763aa15987cb7f1a444` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:38.629815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852a636-01da-44f3-b4ca-9c59d31fedb0 None admin] RBAC: Authorizing `identity:get_user(user_id=5bd0770700d54763aa15987cb7f1a444)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:38.631630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852a636-01da-44f3-b4ca-9c59d31fedb0 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:38.633824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 135/260] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:38 2022] GET /identity/v3/users/5bd0770700d54763aa15987cb7f1a444 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:38.635203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852a636-01da-44f3-b4ca-9c59d31fedb0 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:38.669229 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:38.674626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e2765-ab0f-403b-94dd-f49bde82a188 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:38.722756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c073ca7-2a33-4065-9458-4e23547a45d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:38.723450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c073ca7-2a33-4065-9458-4e23547a45d0 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:38.724119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c073ca7-2a33-4065-9458-4e23547a45d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:38.724582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c073ca7-2a33-4065-9458-4e23547a45d0 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:38.726343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c073ca7-2a33-4065-9458-4e23547a45d0 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:38.728300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c073ca7-2a33-4065-9458-4e23547a45d0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:38.730865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 126/261] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:38 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:38.732012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c073ca7-2a33-4065-9458-4e23547a45d0 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:38.854970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852a636-01da-44f3-b4ca-9c59d31fedb0 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:38.940750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0d042-7f65-4fd6-b3c1-0b0d5dc39ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:38.942196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c073ca7-2a33-4065-9458-4e23547a45d0 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:38.942325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0d042-7f65-4fd6-b3c1-0b0d5dc39ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:38.942725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0d042-7f65-4fd6-b3c1-0b0d5dc39ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:38.943184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0d042-7f65-4fd6-b3c1-0b0d5dc39ff2 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:38.945991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0d042-7f65-4fd6-b3c1-0b0d5dc39ff2 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:38.951427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0d042-7f65-4fd6-b3c1-0b0d5dc39ff2 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:38.955065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 136/262] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:38 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:38.960878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0d042-7f65-4fd6-b3c1-0b0d5dc39ff2 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:38.981492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:39.024230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078ff5f-9c1b-4b2d-ad35-aa712520397d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:39.025379 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078ff5f-9c1b-4b2d-ad35-aa712520397d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:39.025899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078ff5f-9c1b-4b2d-ad35-aa712520397d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:39.026407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078ff5f-9c1b-4b2d-ad35-aa712520397d None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:39.029720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078ff5f-9c1b-4b2d-ad35-aa712520397d None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:39.032610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b078ff5f-9c1b-4b2d-ad35-aa712520397d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:39.042873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 127/263] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:38 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:39.042873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078ff5f-9c1b-4b2d-ad35-aa712520397d None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:39.126386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:39.165749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0d042-7f65-4fd6-b3c1-0b0d5dc39ff2 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:39.242469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b078ff5f-9c1b-4b2d-ad35-aa712520397d None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:39.244509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93bc9-9a9f-40df-9b60-3c23c93c10e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:39.245798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93bc9-9a9f-40df-9b60-3c23c93c10e8 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:39.246272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93bc9-9a9f-40df-9b60-3c23c93c10e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:39.247688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b93bc9-9a9f-40df-9b60-3c23c93c10e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:39.298559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50379528-51c8-4549-be04-f72ec2a27151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:39.300089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50379528-51c8-4549-be04-f72ec2a27151 None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:39.301245 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50379528-51c8-4549-be04-f72ec2a27151 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:39.301726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50379528-51c8-4549-be04-f72ec2a27151 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1/users/96e2c26992bc4dc48f6e11cabb12c284/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:39.304723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50379528-51c8-4549-be04-f72ec2a27151 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac99f487657f41258cd0118165066bb1, user_id=96e2c26992bc4dc48f6e11cabb12c284, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:39.307012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50379528-51c8-4549-be04-f72ec2a27151 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:39.398518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b93bc9-9a9f-40df-9b60-3c23c93c10e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:39.401343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b93bc9-9a9f-40df-9b60-3c23c93c10e8 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:39.415782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 128/264] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:05:39 2022] PUT /identity/v3/projects/ac99f487657f41258cd0118165066bb1/users/96e2c26992bc4dc48f6e11cabb12c284/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:05:39.417994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50379528-51c8-4549-be04-f72ec2a27151 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:39.435484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:39.604928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:39.618216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50379528-51c8-4549-be04-f72ec2a27151 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:39.623716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 137/265] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:39.625805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93bc9-9a9f-40df-9b60-3c23c93c10e8 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:39.691800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cccdb-3a87-4338-adb4-5434e1b7e402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:39.692907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cccdb-3a87-4338-adb4-5434e1b7e402 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:39.693427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cccdb-3a87-4338-adb4-5434e1b7e402 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:39.694201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cccdb-3a87-4338-adb4-5434e1b7e402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:39.695673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cccdb-3a87-4338-adb4-5434e1b7e402 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=ac99f487657f41258cd0118165066bb1, user.id=5bd0770700d54763aa15987cb7f1a444) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:39.696244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cccdb-3a87-4338-adb4-5434e1b7e402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:39.699376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cccdb-3a87-4338-adb4-5434e1b7e402 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:39.710206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 129/266] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:39 2022] GET /identity/v3/role_assignments?user.id=5bd0770700d54763aa15987cb7f1a444&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:39.711474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f6114-91b2-43b6-8fe0-257f26f2ce9e None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:39.711932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f6114-91b2-43b6-8fe0-257f26f2ce9e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:39.715585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f6114-91b2-43b6-8fe0-257f26f2ce9e None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:39.715585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 130/267] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:39.717575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f6114-91b2-43b6-8fe0-257f26f2ce9e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:39.817392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b93bc9-9a9f-40df-9b60-3c23c93c10e8 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:39.847066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:39.899429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e02f34-39b6-482c-900f-b1dcb9db6a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:39.901067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e02f34-39b6-482c-900f-b1dcb9db6a80 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:39.901916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e02f34-39b6-482c-900f-b1dcb9db6a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:39.902632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e02f34-39b6-482c-900f-b1dcb9db6a80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:39.905715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e02f34-39b6-482c-900f-b1dcb9db6a80 None admin] RBAC: Adding query filter params (role.id=558d573b58fa4ba59d56e645fd5a0000, scope.project.id=ac99f487657f41258cd0118165066bb1, user.id=188b456e7b414043a219b9e59266cbf2) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:39.906678 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e02f34-39b6-482c-900f-b1dcb9db6a80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:39.910818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e02f34-39b6-482c-900f-b1dcb9db6a80 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:39.920821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 138/268] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:39 2022] GET /identity/v3/role_assignments?user.id=188b456e7b414043a219b9e59266cbf2&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=558d573b58fa4ba59d56e645fd5a0000 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:39.927593 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2020b7-db1d-4a71-b52f-4b496584f5b1 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:39.928947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2020b7-db1d-4a71-b52f-4b496584f5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:39.929681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2020b7-db1d-4a71-b52f-4b496584f5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:39.980372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ef2020b7-db1d-4a71-b52f-4b496584f5b1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:39.982909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ef2020b7-db1d-4a71-b52f-4b496584f5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:40.104369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f6114-91b2-43b6-8fe0-257f26f2ce9e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:40.188922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 139/269] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:40.201100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a0d24-1979-4c7b-b55f-2264da751642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:40.202113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a0d24-1979-4c7b-b55f-2264da751642 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:40.202542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a0d24-1979-4c7b-b55f-2264da751642 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:40.203046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a0d24-1979-4c7b-b55f-2264da751642 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:40.204662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a0d24-1979-4c7b-b55f-2264da751642 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:40.207281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760a0d24-1979-4c7b-b55f-2264da751642 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:40.210036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 131/270] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:39 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:40.219436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2020b7-db1d-4a71-b52f-4b496584f5b1 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:40.390099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2020b7-db1d-4a71-b52f-4b496584f5b1 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:40.393768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:40.443288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446cc1c-05eb-4032-a598-4e0668b2defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:40.444327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446cc1c-05eb-4032-a598-4e0668b2defe None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:40.444816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446cc1c-05eb-4032-a598-4e0668b2defe None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:40.445195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446cc1c-05eb-4032-a598-4e0668b2defe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:40.446134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446cc1c-05eb-4032-a598-4e0668b2defe None admin] RBAC: Adding query filter params (role.id=b9c95d84eaca4444b2c7b8b5eed61a59, scope.project.id=ac99f487657f41258cd0118165066bb1, user.id=5bd0770700d54763aa15987cb7f1a444) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:40.446627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446cc1c-05eb-4032-a598-4e0668b2defe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:40.448501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8446cc1c-05eb-4032-a598-4e0668b2defe None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:40.461276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 140/271] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:40 2022] GET /identity/v3/role_assignments?user.id=5bd0770700d54763aa15987cb7f1a444&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=b9c95d84eaca4444b2c7b8b5eed61a59 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:40.657527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bf946-40fc-4780-8166-abe251a08a08 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:40.658110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bf946-40fc-4780-8166-abe251a08a08 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:40.658546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bf946-40fc-4780-8166-abe251a08a08 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:40.660391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 132/272] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:05:40.669620 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ca157-88f5-4e2d-8a5c-3f33224308de None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:40.670073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ca157-88f5-4e2d-8a5c-3f33224308de None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:40.670585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ca157-88f5-4e2d-8a5c-3f33224308de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:40.700425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-902ca157-88f5-4e2d-8a5c-3f33224308de None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:40.701896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-902ca157-88f5-4e2d-8a5c-3f33224308de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:40.834706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 141/273] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:40.854814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bf946-40fc-4780-8166-abe251a08a08 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:40.925433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:41.089165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bf946-40fc-4780-8166-abe251a08a08 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:41.161408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4c34d-c2a1-4c44-95fe-2bd0b8298e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:41.163692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4c34d-c2a1-4c44-95fe-2bd0b8298e58 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:41.164399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4c34d-c2a1-4c44-95fe-2bd0b8298e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:41.165074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4c34d-c2a1-4c44-95fe-2bd0b8298e58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:41.189772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4c34d-c2a1-4c44-95fe-2bd0b8298e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-a7b4c34d-c2a1-4c44-95fe-2bd0b8298e58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:41.202179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:41.217409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:41.218925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 133/274] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:41.220197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ca157-88f5-4e2d-8a5c-3f33224308de None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:41.313214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4c34d-c2a1-4c44-95fe-2bd0b8298e58 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:41.404647 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:41.416608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ca157-88f5-4e2d-8a5c-3f33224308de None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:41.486947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:41.491899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c10b36-3acd-4293-aa4a-ba7db1feff01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:41.492780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c10b36-3acd-4293-aa4a-ba7db1feff01 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:41.493279 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c10b36-3acd-4293-aa4a-ba7db1feff01 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:41.493771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c10b36-3acd-4293-aa4a-ba7db1feff01 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:41.495035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c10b36-3acd-4293-aa4a-ba7db1feff01 None admin] RBAC: Adding query filter params (name=member) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:41.495583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c10b36-3acd-4293-aa4a-ba7db1feff01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:41.497792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c10b36-3acd-4293-aa4a-ba7db1feff01 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:41.519741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4c34d-c2a1-4c44-95fe-2bd0b8298e58 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:41.519905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 142/275] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:41.522360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c10b36-3acd-4293-aa4a-ba7db1feff01 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:41.586413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f0b83-cce9-44a1-9d77-632cb8eb29fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:41.587539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0b83-cce9-44a1-9d77-632cb8eb29fe None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:41.588100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0b83-cce9-44a1-9d77-632cb8eb29fe None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:41.589953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f0b83-cce9-44a1-9d77-632cb8eb29fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:41.591368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f0b83-cce9-44a1-9d77-632cb8eb29fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:41.593218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f0b83-cce9-44a1-9d77-632cb8eb29fe None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:41.601371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 134/276] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:05:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:41.606593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f0b83-cce9-44a1-9d77-632cb8eb29fe None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:41.725351 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c10b36-3acd-4293-aa4a-ba7db1feff01 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:41.786561 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996f4c9-27be-4c0c-aebc-4b93c652d9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:41.787830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996f4c9-27be-4c0c-aebc-4b93c652d9ce None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:41.788343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996f4c9-27be-4c0c-aebc-4b93c652d9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:41.788842 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996f4c9-27be-4c0c-aebc-4b93c652d9ce None admin] PATH_INFO: `/v3/users/d2640f0f0e6d418d9c50733ec11570ec` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:41.821497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996f4c9-27be-4c0c-aebc-4b93c652d9ce None admin] RBAC: Authorizing `identity:get_user(user_id=d2640f0f0e6d418d9c50733ec11570ec)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:41.822386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996f4c9-27be-4c0c-aebc-4b93c652d9ce None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:41.826291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 143/277] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:41 2022] GET /identity/v3/users/d2640f0f0e6d418d9c50733ec11570ec => generated 293 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:41.838141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996f4c9-27be-4c0c-aebc-4b93c652d9ce None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:41.840719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f0b83-cce9-44a1-9d77-632cb8eb29fe None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:41.883845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:41.915703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf1dff-2932-47d7-9f5e-add1d5fb8507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:41.917136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf1dff-2932-47d7-9f5e-add1d5fb8507 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:41.917886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf1dff-2932-47d7-9f5e-add1d5fb8507 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:41.918594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf1dff-2932-47d7-9f5e-add1d5fb8507 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:41.920301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf1dff-2932-47d7-9f5e-add1d5fb8507 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:41.924165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf1dff-2932-47d7-9f5e-add1d5fb8507 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:42.046725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996f4c9-27be-4c0c-aebc-4b93c652d9ce None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:42.119252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c61476-ce28-4108-b001-047c18520305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:42.120441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61476-ce28-4108-b001-047c18520305 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:42.120960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61476-ce28-4108-b001-047c18520305 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:42.121456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c61476-ce28-4108-b001-047c18520305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:42.122946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c61476-ce28-4108-b001-047c18520305 None admin] RBAC: Adding query filter params (role.id=e5a65ae2d0884e419cdbc70c9223eac2, user.id=d2640f0f0e6d418d9c50733ec11570ec, scope.system=all) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:42.123492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c61476-ce28-4108-b001-047c18520305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:42.126396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c61476-ce28-4108-b001-047c18520305 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:42.144457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 144/278] 10.222.0.64 () {56 vars in 1308 bytes} [Fri Sep 23 01:05:41 2022] GET /identity/v3/role_assignments?user.id=d2640f0f0e6d418d9c50733ec11570ec&scope.system=all&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:42.151286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 135/279] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:05:41 2022] POST /identity/v3/users => generated 293 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:05:43.446531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886dc60-b237-4074-a4b9-75f7b76f0048 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:43.448085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886dc60-b237-4074-a4b9-75f7b76f0048 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:43.448907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886dc60-b237-4074-a4b9-75f7b76f0048 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:43.451224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 145/280] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:43.461394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77074c87-5cd4-4fe5-af6c-fe75e7870b93 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:43.461999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77074c87-5cd4-4fe5-af6c-fe75e7870b93 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:43.462362 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77074c87-5cd4-4fe5-af6c-fe75e7870b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:43.503749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-77074c87-5cd4-4fe5-af6c-fe75e7870b93 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:43.505769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-77074c87-5cd4-4fe5-af6c-fe75e7870b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:43.513461 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:43.689098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 136/281] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:44.036055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886dc60-b237-4074-a4b9-75f7b76f0048 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:44.142575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e7092-d432-4a97-9050-8a18990e291b None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:44.143087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e7092-d432-4a97-9050-8a18990e291b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:44.143491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e7092-d432-4a97-9050-8a18990e291b None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:44.150700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 137/282] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:44.156958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff053b-e5cf-4d56-a978-f6b034731131 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:44.157408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff053b-e5cf-4d56-a978-f6b034731131 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:44.157875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff053b-e5cf-4d56-a978-f6b034731131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:44.213357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-5dff053b-e5cf-4d56-a978-f6b034731131 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:44.215309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5dff053b-e5cf-4d56-a978-f6b034731131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:44.227793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:44.273539 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886dc60-b237-4074-a4b9-75f7b76f0048 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:44.433925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb27c2-270a-4f7b-8f75-fc488f80e1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:44.434926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb27c2-270a-4f7b-8f75-fc488f80e1dd None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:44.435527 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb27c2-270a-4f7b-8f75-fc488f80e1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:44.435930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb27c2-270a-4f7b-8f75-fc488f80e1dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:44.446084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 138/283] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:44.468583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb27c2-270a-4f7b-8f75-fc488f80e1dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-cffb27c2-270a-4f7b-8f75-fc488f80e1dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:44.480302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:44.482067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:44.483600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:44.488479 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 146/284] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 454 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:44.495807 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff053b-e5cf-4d56-a978-f6b034731131 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:44.648220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:44.726471 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff053b-e5cf-4d56-a978-f6b034731131 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:44.829906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662e939-84cd-4282-a0e9-aa3a8b2589ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:44.830941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662e939-84cd-4282-a0e9-aa3a8b2589ff None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:44.832033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662e939-84cd-4282-a0e9-aa3a8b2589ff None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:44.832519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f662e939-84cd-4282-a0e9-aa3a8b2589ff None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:44.834136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662e939-84cd-4282-a0e9-aa3a8b2589ff None admin] RBAC: Adding query filter params (name=member) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:44.834273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662e939-84cd-4282-a0e9-aa3a8b2589ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:44.838905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f662e939-84cd-4282-a0e9-aa3a8b2589ff None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:44.861775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 139/285] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:44.870700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb27c2-270a-4f7b-8f75-fc488f80e1dd None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:45.056874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662e939-84cd-4282-a0e9-aa3a8b2589ff None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:45.135172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb27c2-270a-4f7b-8f75-fc488f80e1dd None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:45.256594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e8060-ab40-4936-9890-ed01454a9f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:45.263734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e8060-ab40-4936-9890-ed01454a9f2e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:45.264674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e8060-ab40-4936-9890-ed01454a9f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:45.265507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e8060-ab40-4936-9890-ed01454a9f2e None admin] PATH_INFO: `/v3/users/96e2c26992bc4dc48f6e11cabb12c284` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:45.265787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f662e939-84cd-4282-a0e9-aa3a8b2589ff None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:45.268478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e8060-ab40-4936-9890-ed01454a9f2e None admin] RBAC: Authorizing `identity:get_user(user_id=96e2c26992bc4dc48f6e11cabb12c284)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:45.279962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e8060-ab40-4936-9890-ed01454a9f2e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:45.284224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 147/286] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:44 2022] GET /identity/v3/users/96e2c26992bc4dc48f6e11cabb12c284 => generated 297 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:45.302332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e8060-ab40-4936-9890-ed01454a9f2e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:45.322626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:45.337606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b2ab4-65a3-477f-9f87-737b1c5a21f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:45.338129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:45.338628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2ab4-65a3-477f-9f87-737b1c5a21f8 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:45.338910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2ab4-65a3-477f-9f87-737b1c5a21f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:45.339185 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b2ab4-65a3-477f-9f87-737b1c5a21f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:45.373034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b2ab4-65a3-477f-9f87-737b1c5a21f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-b54b2ab4-65a3-477f-9f87-737b1c5a21f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:45.386252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:45.388040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:45.389546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 140/287] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:45.401831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b2ab4-65a3-477f-9f87-737b1c5a21f8 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:45.614935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b2ab4-65a3-477f-9f87-737b1c5a21f8 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:45.692563 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ba4c3-eb15-48e6-87aa-ea41f769b9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:45.694368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ba4c3-eb15-48e6-87aa-ea41f769b9fa None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:45.695001 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ba4c3-eb15-48e6-87aa-ea41f769b9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:45.695637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ba4c3-eb15-48e6-87aa-ea41f769b9fa None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:45.697154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ba4c3-eb15-48e6-87aa-ea41f769b9fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:45.697931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ba4c3-eb15-48e6-87aa-ea41f769b9fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:45.699282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e8060-ab40-4936-9890-ed01454a9f2e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:45.700922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ba4c3-eb15-48e6-87aa-ea41f769b9fa None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:45.725638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 141/288] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:45.735574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ba4c3-eb15-48e6-87aa-ea41f769b9fa None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:45.849541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797b553-4c01-4cb2-a5b9-5ad0e8c6b70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:45.850650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b553-4c01-4cb2-a5b9-5ad0e8c6b70e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:45.851145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b553-4c01-4cb2-a5b9-5ad0e8c6b70e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:45.855369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b553-4c01-4cb2-a5b9-5ad0e8c6b70e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:45.913288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:46.042919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ba4c3-eb15-48e6-87aa-ea41f769b9fa None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:46.130703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797b553-4c01-4cb2-a5b9-5ad0e8c6b70e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:46.133654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797b553-4c01-4cb2-a5b9-5ad0e8c6b70e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:46.180232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a10ac5-d754-4b51-a693-7dcfd45ef970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:46.181221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a10ac5-d754-4b51-a693-7dcfd45ef970 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:46.181693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a10ac5-d754-4b51-a693-7dcfd45ef970 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:46.182156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a10ac5-d754-4b51-a693-7dcfd45ef970 None admin] PATH_INFO: `/v3/users/188b456e7b414043a219b9e59266cbf2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:46.185960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a10ac5-d754-4b51-a693-7dcfd45ef970 None admin] RBAC: Authorizing `identity:get_user(user_id=188b456e7b414043a219b9e59266cbf2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:46.188179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:46.188585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a10ac5-d754-4b51-a693-7dcfd45ef970 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:46.191810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 142/289] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:45 2022] GET /identity/v3/users/188b456e7b414043a219b9e59266cbf2 => generated 297 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:46.204747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a10ac5-d754-4b51-a693-7dcfd45ef970 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:46.431069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 148/290] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1130 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 01:05:46.433925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b665d-b397-42b1-a4f5-5264a62ce38c None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:46.435071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b665d-b397-42b1-a4f5-5264a62ce38c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:46.435986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b665d-b397-42b1-a4f5-5264a62ce38c None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:46.444058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 149/291] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:46.446206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b665d-b397-42b1-a4f5-5264a62ce38c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:46.460594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a10ac5-d754-4b51-a693-7dcfd45ef970 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:46.500596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:46.517135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6d6d7-e0aa-40a9-b78a-c59e2f995256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:46.518030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6d6d7-e0aa-40a9-b78a-c59e2f995256 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:46.521153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6d6d7-e0aa-40a9-b78a-c59e2f995256 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:46.521153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6d6d7-e0aa-40a9-b78a-c59e2f995256 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:46.521153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6d6d7-e0aa-40a9-b78a-c59e2f995256 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:46.522463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6d6d7-e0aa-40a9-b78a-c59e2f995256 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:46.529951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 143/292] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:46 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:46.532496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b8144-d03d-4477-87c5-8916e8e04126 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:46.532696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b8144-d03d-4477-87c5-8916e8e04126 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:46.533213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b8144-d03d-4477-87c5-8916e8e04126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:46.559782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-355b8144-d03d-4477-87c5-8916e8e04126 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:46.561494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-355b8144-d03d-4477-87c5-8916e8e04126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:46.629686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b665d-b397-42b1-a4f5-5264a62ce38c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:46.651741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:46.687060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1547dc9-5380-43a4-8653-1d469e0ff687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:46.687900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547dc9-5380-43a4-8653-1d469e0ff687 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:46.688151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547dc9-5380-43a4-8653-1d469e0ff687 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:46.688485 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1547dc9-5380-43a4-8653-1d469e0ff687 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:46.691596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1547dc9-5380-43a4-8653-1d469e0ff687 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:46.694028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1547dc9-5380-43a4-8653-1d469e0ff687 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:46.696113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 150/293] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:46 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:46.697498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d0e72-c3de-4625-9a77-20b8577f542c None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:46.697971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d0e72-c3de-4625-9a77-20b8577f542c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:46.708121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d0e72-c3de-4625-9a77-20b8577f542c None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:46.708121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 151/294] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:46.708121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d0e72-c3de-4625-9a77-20b8577f542c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:46.709210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 144/295] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:46.709210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b8144-d03d-4477-87c5-8916e8e04126 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:46.855395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d0e72-c3de-4625-9a77-20b8577f542c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:46.906487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b8144-d03d-4477-87c5-8916e8e04126 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:46.920210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e270496-8d03-49f6-b500-3f894d9f37ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:46.921084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e270496-8d03-49f6-b500-3f894d9f37ac None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:46.921385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e270496-8d03-49f6-b500-3f894d9f37ac None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:46.921666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e270496-8d03-49f6-b500-3f894d9f37ac None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1/users/188b456e7b414043a219b9e59266cbf2/roles/558d573b58fa4ba59d56e645fd5a0000` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:46.923828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e270496-8d03-49f6-b500-3f894d9f37ac None admin] RBAC: Authorizing `identity:create_grant(project_id=ac99f487657f41258cd0118165066bb1, user_id=188b456e7b414043a219b9e59266cbf2, role_id=558d573b58fa4ba59d56e645fd5a0000)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:46.926429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e270496-8d03-49f6-b500-3f894d9f37ac None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:46.943938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:46.960466 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f312a-e1a5-4276-aad2-cac63b0bcf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:46.961202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f312a-e1a5-4276-aad2-cac63b0bcf59 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:46.962755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f312a-e1a5-4276-aad2-cac63b0bcf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:46.963022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f312a-e1a5-4276-aad2-cac63b0bcf59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:46.963996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f312a-e1a5-4276-aad2-cac63b0bcf59 None admin] RBAC: Adding query filter params (role.id=e5a65ae2d0884e419cdbc70c9223eac2, scope.project.id=ac99f487657f41258cd0118165066bb1, user.id=96e2c26992bc4dc48f6e11cabb12c284) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:46.964309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f312a-e1a5-4276-aad2-cac63b0bcf59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:46.966181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f312a-e1a5-4276-aad2-cac63b0bcf59 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:46.977121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 145/296] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:46 2022] GET /identity/v3/role_assignments?user.id=96e2c26992bc4dc48f6e11cabb12c284&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:46.978479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb693419-3c78-4ad8-93b9-2f6aeb4b694d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:46.978782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb693419-3c78-4ad8-93b9-2f6aeb4b694d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:46.979072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb693419-3c78-4ad8-93b9-2f6aeb4b694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:47.007818 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-bb693419-3c78-4ad8-93b9-2f6aeb4b694d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:47.009278 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 152/297] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:05:46 2022] PUT /identity/v3/projects/ac99f487657f41258cd0118165066bb1/users/188b456e7b414043a219b9e59266cbf2/roles/558d573b58fa4ba59d56e645fd5a0000 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:05:47.011128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-bb693419-3c78-4ad8-93b9-2f6aeb4b694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:47.012834 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784e8be-3635-4e3c-9631-ef3fd33f1966 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:47.013832 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784e8be-3635-4e3c-9631-ef3fd33f1966 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:47.014611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784e8be-3635-4e3c-9631-ef3fd33f1966 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:47.020251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 153/298] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:47.031698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dec83f-3ba4-4dd0-b1a3-70a134ce672d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:47.032671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dec83f-3ba4-4dd0-b1a3-70a134ce672d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:47.033698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dec83f-3ba4-4dd0-b1a3-70a134ce672d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:47.079461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-90dec83f-3ba4-4dd0-b1a3-70a134ce672d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:47.083251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-90dec83f-3ba4-4dd0-b1a3-70a134ce672d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:47.189649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:47.246544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 146/299] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:47.251964 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb693419-3c78-4ad8-93b9-2f6aeb4b694d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:47.294351 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 154/300] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:47.485880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb693419-3c78-4ad8-93b9-2f6aeb4b694d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:47.513803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:47.555315 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392b757-66d1-4947-a948-4761222af4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:47.556468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392b757-66d1-4947-a948-4761222af4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:47.557076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392b757-66d1-4947-a948-4761222af4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:47.557524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392b757-66d1-4947-a948-4761222af4e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:47.577633 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3392b757-66d1-4947-a948-4761222af4e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-3392b757-66d1-4947-a948-4761222af4e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:47.583675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:47.584940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:47.585834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:47.587876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 147/301] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:47.596453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dec83f-3ba4-4dd0-b1a3-70a134ce672d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:47.735874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392b757-66d1-4947-a948-4761222af4e3 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:47.838732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dec83f-3ba4-4dd0-b1a3-70a134ce672d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:47.912246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aaa49-7aaa-4de6-89e0-c25abbeaba15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:47.913331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aaa49-7aaa-4de6-89e0-c25abbeaba15 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:47.913821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aaa49-7aaa-4de6-89e0-c25abbeaba15 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:47.914300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aaa49-7aaa-4de6-89e0-c25abbeaba15 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:47.915916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50aaa49-7aaa-4de6-89e0-c25abbeaba15 None admin] RBAC: Adding query filter params (name=member) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:47.916467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50aaa49-7aaa-4de6-89e0-c25abbeaba15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:47.919613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50aaa49-7aaa-4de6-89e0-c25abbeaba15 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:47.937128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 155/302] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:47.946536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aaa49-7aaa-4de6-89e0-c25abbeaba15 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:47.982696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392b757-66d1-4947-a948-4761222af4e3 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:47.995375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:48.056957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689adc2f-ae34-46ec-b307-eb29137154a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:48.057930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689adc2f-ae34-46ec-b307-eb29137154a2 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:48.058400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689adc2f-ae34-46ec-b307-eb29137154a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:48.058855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689adc2f-ae34-46ec-b307-eb29137154a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:48.060990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689adc2f-ae34-46ec-b307-eb29137154a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:48.063764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689adc2f-ae34-46ec-b307-eb29137154a2 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:48.067034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 148/303] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:05:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:48.068628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689adc2f-ae34-46ec-b307-eb29137154a2 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:48.111341 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aaa49-7aaa-4de6-89e0-c25abbeaba15 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:48.160107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:48.185817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ccbc-71ac-41b9-8351-5af216e8d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:48.186903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ccbc-71ac-41b9-8351-5af216e8d655 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:48.187397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ccbc-71ac-41b9-8351-5af216e8d655 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:48.187928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ccbc-71ac-41b9-8351-5af216e8d655 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:48.212552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86ccbc-71ac-41b9-8351-5af216e8d655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-fd86ccbc-71ac-41b9-8351-5af216e8d655 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:48.227068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:48.228952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:48.231375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 156/304] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:48.232695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ccbc-71ac-41b9-8351-5af216e8d655 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:48.258802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689adc2f-ae34-46ec-b307-eb29137154a2 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:48.328441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b87345-64ea-4a1f-a57d-000897184790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:48.329495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b87345-64ea-4a1f-a57d-000897184790 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:48.329922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b87345-64ea-4a1f-a57d-000897184790 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:48.330380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b87345-64ea-4a1f-a57d-000897184790 None admin] PATH_INFO: `/v3/users/d2640f0f0e6d418d9c50733ec11570ec` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:48.332557 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b87345-64ea-4a1f-a57d-000897184790 None admin] RBAC: Authorizing `identity:get_user(user_id=d2640f0f0e6d418d9c50733ec11570ec)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:48.335342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b87345-64ea-4a1f-a57d-000897184790 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:48.338687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 149/305] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:48 2022] GET /identity/v3/users/d2640f0f0e6d418d9c50733ec11570ec => generated 293 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:48.339940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b87345-64ea-4a1f-a57d-000897184790 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:48.425688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:48.443083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ccbc-71ac-41b9-8351-5af216e8d655 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:48.511240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3ceef-9b2e-414b-9ac9-2a67ccb01557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:48.512346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3ceef-9b2e-414b-9ac9-2a67ccb01557 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:48.512835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3ceef-9b2e-414b-9ac9-2a67ccb01557 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:48.513280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3ceef-9b2e-414b-9ac9-2a67ccb01557 None admin] PATH_INFO: `/v3/groups` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:48.514441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3ceef-9b2e-414b-9ac9-2a67ccb01557 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:48.517074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3ceef-9b2e-414b-9ac9-2a67ccb01557 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:48.549329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b87345-64ea-4a1f-a57d-000897184790 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:48.573337 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:48.605358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 157/306] 10.222.0.64 () {60 vars in 1134 bytes} [Fri Sep 23 01:05:48 2022] POST /identity/v3/groups => generated 229 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:05:48.606602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3ceef-9b2e-414b-9ac9-2a67ccb01557 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:48.633794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b766d-ee26-4d05-a10d-a3fa25ba3944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:48.634533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b766d-ee26-4d05-a10d-a3fa25ba3944 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:48.634812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b766d-ee26-4d05-a10d-a3fa25ba3944 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:48.635084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b766d-ee26-4d05-a10d-a3fa25ba3944 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:48.637039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b766d-ee26-4d05-a10d-a3fa25ba3944 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:48.637039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b766d-ee26-4d05-a10d-a3fa25ba3944 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:48.637649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b766d-ee26-4d05-a10d-a3fa25ba3944 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:48.650300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 150/307] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:48.666104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b766d-ee26-4d05-a10d-a3fa25ba3944 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:48.832785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3ceef-9b2e-414b-9ac9-2a67ccb01557 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:48.881370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93cab5-605a-49f4-8ac4-f9e710d15b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:48.882282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93cab5-605a-49f4-8ac4-f9e710d15b79 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:48.882745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93cab5-605a-49f4-8ac4-f9e710d15b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:48.883322 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c93cab5-605a-49f4-8ac4-f9e710d15b79 None admin] PATH_INFO: `/v3/system/users/d2640f0f0e6d418d9c50733ec11570ec/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:48.885194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93cab5-605a-49f4-8ac4-f9e710d15b79 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d2640f0f0e6d418d9c50733ec11570ec, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:48.887534 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c93cab5-605a-49f4-8ac4-f9e710d15b79 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:48.913829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b766d-ee26-4d05-a10d-a3fa25ba3944 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:48.977567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 158/308] 10.222.0.64 () {58 vars in 1334 bytes} [Fri Sep 23 01:05:48 2022] PUT /identity/v3/system/users/d2640f0f0e6d418d9c50733ec11570ec/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:05:48.984154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7a35b-8317-406f-915e-bbe2f8cb9295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:48.985411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7a35b-8317-406f-915e-bbe2f8cb9295 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:48.985888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7a35b-8317-406f-915e-bbe2f8cb9295 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:48.986356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7a35b-8317-406f-915e-bbe2f8cb9295 None admin] PATH_INFO: `/v3/users/aaa76e79ab1347a9834648c82a07abe1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:49.015727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7a35b-8317-406f-915e-bbe2f8cb9295 None admin] RBAC: Authorizing `identity:get_user(user_id=aaa76e79ab1347a9834648c82a07abe1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:49.020033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7a35b-8317-406f-915e-bbe2f8cb9295 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:49.024170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 151/309] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:48 2022] GET /identity/v3/users/aaa76e79ab1347a9834648c82a07abe1 => generated 293 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:49.037417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93cab5-605a-49f4-8ac4-f9e710d15b79 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:49.077056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:49.267979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c93cab5-605a-49f4-8ac4-f9e710d15b79 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:49.336660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3a636-6929-4485-b0ed-47872bf75799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:49.337760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3a636-6929-4485-b0ed-47872bf75799 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:49.338263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3a636-6929-4485-b0ed-47872bf75799 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:49.338757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3a636-6929-4485-b0ed-47872bf75799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:49.340273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3a636-6929-4485-b0ed-47872bf75799 None admin] RBAC: Adding query filter params (role.id=558d573b58fa4ba59d56e645fd5a0000, user.id=aaa76e79ab1347a9834648c82a07abe1, scope.system=all) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:49.340815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3a636-6929-4485-b0ed-47872bf75799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:49.343885 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d3a636-6929-4485-b0ed-47872bf75799 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:49.354946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 159/310] 10.222.0.64 () {56 vars in 1308 bytes} [Fri Sep 23 01:05:49 2022] GET /identity/v3/role_assignments?user.id=aaa76e79ab1347a9834648c82a07abe1&scope.system=all&role.id=558d573b58fa4ba59d56e645fd5a0000 => generated 226 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:50.344514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ecfa-c399-4a30-ab5a-015ad7925194 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:50.345616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ecfa-c399-4a30-ab5a-015ad7925194 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:50.346189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ecfa-c399-4a30-ab5a-015ad7925194 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:50.348671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 152/311] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:05:50.356096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b45aef-15f7-4939-9064-db829d374dda None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:50.357161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b45aef-15f7-4939-9064-db829d374dda None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:50.357657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b45aef-15f7-4939-9064-db829d374dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:50.383536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d8b45aef-15f7-4939-9064-db829d374dda None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:50.385944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d8b45aef-15f7-4939-9064-db829d374dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:50.499704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 160/312] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:50.982043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ecfa-c399-4a30-ab5a-015ad7925194 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:51.043196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:51.186386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6ecfa-c399-4a30-ab5a-015ad7925194 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:51.257129 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dcaf34-d19e-48f1-bfe1-f15fac632be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:51.258322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcaf34-d19e-48f1-bfe1-f15fac632be7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:51.258811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcaf34-d19e-48f1-bfe1-f15fac632be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:51.259292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcaf34-d19e-48f1-bfe1-f15fac632be7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:51.281444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dcaf34-d19e-48f1-bfe1-f15fac632be7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-b1dcaf34-d19e-48f1-bfe1-f15fac632be7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:51.291704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:51.293518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:51.295974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 153/313] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:51.303531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b45aef-15f7-4939-9064-db829d374dda None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:51.324683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:51.512549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b45aef-15f7-4939-9064-db829d374dda None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:51.575444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac12f7a-b8d2-4f57-9286-56ad5bde9480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:51.576484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12f7a-b8d2-4f57-9286-56ad5bde9480 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:51.576963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12f7a-b8d2-4f57-9286-56ad5bde9480 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:51.577428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac12f7a-b8d2-4f57-9286-56ad5bde9480 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:51.578494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac12f7a-b8d2-4f57-9286-56ad5bde9480 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:51.578955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac12f7a-b8d2-4f57-9286-56ad5bde9480 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:51.581912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac12f7a-b8d2-4f57-9286-56ad5bde9480 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:51.597685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 161/314] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:51.606259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dcaf34-d19e-48f1-bfe1-f15fac632be7 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:51.802140 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:51.821507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dcaf34-d19e-48f1-bfe1-f15fac632be7 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:51.891593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb756e-13aa-4761-91ca-086b9248880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:51.892748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb756e-13aa-4761-91ca-086b9248880d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:51.893301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb756e-13aa-4761-91ca-086b9248880d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:51.893828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb756e-13aa-4761-91ca-086b9248880d None admin] PATH_INFO: `/v3/users/188b456e7b414043a219b9e59266cbf2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:51.896056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb756e-13aa-4761-91ca-086b9248880d None admin] RBAC: Authorizing `identity:get_user(user_id=188b456e7b414043a219b9e59266cbf2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:51.898830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb756e-13aa-4761-91ca-086b9248880d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:51.902040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 154/315] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:51 2022] GET /identity/v3/users/188b456e7b414043a219b9e59266cbf2 => generated 297 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:51.911198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac12f7a-b8d2-4f57-9286-56ad5bde9480 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:52.059452 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:52.121632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac12f7a-b8d2-4f57-9286-56ad5bde9480 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:52.190255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb19af-300e-4642-8671-b90162f48865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:52.191384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb19af-300e-4642-8671-b90162f48865 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:52.191934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb19af-300e-4642-8671-b90162f48865 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:52.192445 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb19af-300e-4642-8671-b90162f48865 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:52.267817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36f186-8aed-4b16-b838-ba7fe8a76d80 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:52.269324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36f186-8aed-4b16-b838-ba7fe8a76d80 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:52.270113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36f186-8aed-4b16-b838-ba7fe8a76d80 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:52.273435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 155/316] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:52.282935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9371945-1a41-445d-a6e4-f25153379912 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:52.283860 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9371945-1a41-445d-a6e4-f25153379912 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:52.284594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9371945-1a41-445d-a6e4-f25153379912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:52.321029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f9371945-1a41-445d-a6e4-f25153379912 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:52.324968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f9371945-1a41-445d-a6e4-f25153379912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:52.335211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb19af-300e-4642-8671-b90162f48865 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:52.338231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb19af-300e-4642-8671-b90162f48865 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:52.480305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 156/317] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:52.500792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:52.574958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 162/318] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:05:52.584618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9371945-1a41-445d-a6e4-f25153379912 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:52.693393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d158a-b79c-43ee-85d3-f32c3a7ebff3 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:52.694039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d158a-b79c-43ee-85d3-f32c3a7ebff3 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:52.694428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d158a-b79c-43ee-85d3-f32c3a7ebff3 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:52.696955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 163/319] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:05:52.705100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22d651-49ec-4036-8319-7701ff6bf43c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:52.705494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22d651-49ec-4036-8319-7701ff6bf43c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:52.705881 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22d651-49ec-4036-8319-7701ff6bf43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:52.736126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-eb22d651-49ec-4036-8319-7701ff6bf43c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:52.737882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-eb22d651-49ec-4036-8319-7701ff6bf43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:52.770585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:52.800642 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9371945-1a41-445d-a6e4-f25153379912 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:52.888235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a6577-c486-4da4-ba53-810a2d4e3481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:52.889464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a6577-c486-4da4-ba53-810a2d4e3481 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:52.889895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a6577-c486-4da4-ba53-810a2d4e3481 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:52.890399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a6577-c486-4da4-ba53-810a2d4e3481 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:52.892676 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a6577-c486-4da4-ba53-810a2d4e3481 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:52.896194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a6577-c486-4da4-ba53-810a2d4e3481 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:52.899898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 157/320] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:05:52 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:52.901367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a6577-c486-4da4-ba53-810a2d4e3481 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:52.916486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 164/321] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:52.919700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22d651-49ec-4036-8319-7701ff6bf43c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:53.084789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:53.090975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22d651-49ec-4036-8319-7701ff6bf43c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:53.122700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a6577-c486-4da4-ba53-810a2d4e3481 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:53.156510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79956b3-95cb-44cc-934f-dcf90057d7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:53.157589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79956b3-95cb-44cc-934f-dcf90057d7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:53.158066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79956b3-95cb-44cc-934f-dcf90057d7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:53.158609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79956b3-95cb-44cc-934f-dcf90057d7b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:53.160661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79956b3-95cb-44cc-934f-dcf90057d7b9 None admin] RBAC: Adding query filter params (role.id=558d573b58fa4ba59d56e645fd5a0000, scope.project.id=ac99f487657f41258cd0118165066bb1, user.id=188b456e7b414043a219b9e59266cbf2) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:53.161288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79956b3-95cb-44cc-934f-dcf90057d7b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:53.164332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79956b3-95cb-44cc-934f-dcf90057d7b9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:53.183631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 165/322] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:05:52 2022] GET /identity/v3/role_assignments?user.id=188b456e7b414043a219b9e59266cbf2&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=558d573b58fa4ba59d56e645fd5a0000 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:53.183631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16824e0b-955e-48de-98ef-371f29f0b2ec None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:53.183631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16824e0b-955e-48de-98ef-371f29f0b2ec None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:53.183631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16824e0b-955e-48de-98ef-371f29f0b2ec None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:53.183631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 166/323] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:53.190517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129561c-7d49-4d73-bcec-b6d851ff4180 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:53.191151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129561c-7d49-4d73-bcec-b6d851ff4180 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:53.191786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129561c-7d49-4d73-bcec-b6d851ff4180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:53.192901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2f90-d53c-4e22-a7ce-9b1aa4a0cfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:53.194096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2f90-d53c-4e22-a7ce-9b1aa4a0cfdd None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:53.194587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2f90-d53c-4e22-a7ce-9b1aa4a0cfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:53.195100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c2f90-d53c-4e22-a7ce-9b1aa4a0cfdd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:53.197562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c2f90-d53c-4e22-a7ce-9b1aa4a0cfdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:53.199098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c2f90-d53c-4e22-a7ce-9b1aa4a0cfdd None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:53.200937 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 158/324] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:05:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:53.209181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2f90-d53c-4e22-a7ce-9b1aa4a0cfdd None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:53.219760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b129561c-7d49-4d73-bcec-b6d851ff4180 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:53.221045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b129561c-7d49-4d73-bcec-b6d851ff4180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:53.271985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:53.366086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 167/325] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:05:53.367369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129561c-7d49-4d73-bcec-b6d851ff4180 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:53.440445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c2f90-d53c-4e22-a7ce-9b1aa4a0cfdd None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:53.503787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35511df-36b4-455a-9eef-b349d0ddd961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:53.505327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35511df-36b4-455a-9eef-b349d0ddd961 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:53.506118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35511df-36b4-455a-9eef-b349d0ddd961 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:53.506895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35511df-36b4-455a-9eef-b349d0ddd961 None admin] PATH_INFO: `/v3/groups` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:53.508422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35511df-36b4-455a-9eef-b349d0ddd961 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:53.511518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35511df-36b4-455a-9eef-b349d0ddd961 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:53.578446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:53.590622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129561c-7d49-4d73-bcec-b6d851ff4180 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:53.640556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b0ce0-f266-433b-a71f-c19294d014b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:53.641421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b0ce0-f266-433b-a71f-c19294d014b1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:53.641704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b0ce0-f266-433b-a71f-c19294d014b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:53.641983 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b0ce0-f266-433b-a71f-c19294d014b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:53.655421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b0ce0-f266-433b-a71f-c19294d014b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-fd5b0ce0-f266-433b-a71f-c19294d014b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:53.661148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:53.662857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:05:53.664288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 168/326] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:05:53.671702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b0ce0-f266-433b-a71f-c19294d014b1 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:53.682589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 159/327] 10.222.0.64 () {60 vars in 1134 bytes} [Fri Sep 23 01:05:53 2022] POST /identity/v3/groups => generated 226 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:05:53.707142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35511df-36b4-455a-9eef-b349d0ddd961 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:53.862844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b0ce0-f266-433b-a71f-c19294d014b1 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:53.876729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:53.908409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35511df-36b4-455a-9eef-b349d0ddd961 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:53.929156 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40d074-a71b-4f42-9ceb-5cf1efe1e654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:53.930141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40d074-a71b-4f42-9ceb-5cf1efe1e654 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:53.930565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40d074-a71b-4f42-9ceb-5cf1efe1e654 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:53.930979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40d074-a71b-4f42-9ceb-5cf1efe1e654 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:53.932131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40d074-a71b-4f42-9ceb-5cf1efe1e654 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:53.932716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40d074-a71b-4f42-9ceb-5cf1efe1e654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:53.935024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40d074-a71b-4f42-9ceb-5cf1efe1e654 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:53.951851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 169/328] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:53.960993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40d074-a71b-4f42-9ceb-5cf1efe1e654 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:53.981712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f75722-0c5d-4605-9f82-e169a01da300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:53.982430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75722-0c5d-4605-9f82-e169a01da300 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:53.983383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75722-0c5d-4605-9f82-e169a01da300 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:53.985607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75722-0c5d-4605-9f82-e169a01da300 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:53.996952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:54.008085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f75722-0c5d-4605-9f82-e169a01da300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-a8f75722-0c5d-4605-9f82-e169a01da300 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:05:54.021957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:05:54.023969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:05:54.025599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 160/329] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:05:54.032120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f75722-0c5d-4605-9f82-e169a01da300 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:54.181906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40d074-a71b-4f42-9ceb-5cf1efe1e654 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:54.249830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f75722-0c5d-4605-9f82-e169a01da300 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:54.267928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5753a-7c8d-4fd3-96d5-6b807d33f4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:54.269595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5753a-7c8d-4fd3-96d5-6b807d33f4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:54.270400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5753a-7c8d-4fd3-96d5-6b807d33f4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:54.271157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df5753a-7c8d-4fd3-96d5-6b807d33f4f1 None admin] PATH_INFO: `/v3/users/aaa76e79ab1347a9834648c82a07abe1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:54.273781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5753a-7c8d-4fd3-96d5-6b807d33f4f1 None admin] RBAC: Authorizing `identity:get_user(user_id=aaa76e79ab1347a9834648c82a07abe1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:54.277134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df5753a-7c8d-4fd3-96d5-6b807d33f4f1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:54.287710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 170/330] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:53 2022] GET /identity/v3/users/aaa76e79ab1347a9834648c82a07abe1 => generated 293 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:54.291201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5753a-7c8d-4fd3-96d5-6b807d33f4f1 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:54.323430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f15794-0597-43b3-9598-8d270cf0c5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:54.325518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f15794-0597-43b3-9598-8d270cf0c5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:54.325518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f15794-0597-43b3-9598-8d270cf0c5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:54.325518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f15794-0597-43b3-9598-8d270cf0c5a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:54.326810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f15794-0597-43b3-9598-8d270cf0c5a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:54.327371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f15794-0597-43b3-9598-8d270cf0c5a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:54.330290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f15794-0597-43b3-9598-8d270cf0c5a6 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:54.345767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 161/331] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:05:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:54.359426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f15794-0597-43b3-9598-8d270cf0c5a6 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:54.381748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:54.492120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:54.516502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df5753a-7c8d-4fd3-96d5-6b807d33f4f1 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:54.575322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f15794-0597-43b3-9598-8d270cf0c5a6 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:54.591923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51bcee-2a79-41cd-add5-968aa7c1f613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:54.593076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51bcee-2a79-41cd-add5-968aa7c1f613 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:54.593741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51bcee-2a79-41cd-add5-968aa7c1f613 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:54.594250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51bcee-2a79-41cd-add5-968aa7c1f613 None admin] PATH_INFO: `/v3/system/users/aaa76e79ab1347a9834648c82a07abe1/roles/558d573b58fa4ba59d56e645fd5a0000` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:54.597270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51bcee-2a79-41cd-add5-968aa7c1f613 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=aaa76e79ab1347a9834648c82a07abe1, role_id=558d573b58fa4ba59d56e645fd5a0000)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:54.600169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51bcee-2a79-41cd-add5-968aa7c1f613 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:54.634675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ef553-3993-4d82-bfc9-49b597d257bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:54.635708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef553-3993-4d82-bfc9-49b597d257bc None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:54.636188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef553-3993-4d82-bfc9-49b597d257bc None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:54.636680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ef553-3993-4d82-bfc9-49b597d257bc None admin] PATH_INFO: `/v3/users/d2640f0f0e6d418d9c50733ec11570ec` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:54.639035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ef553-3993-4d82-bfc9-49b597d257bc None admin] RBAC: Authorizing `identity:get_user(user_id=d2640f0f0e6d418d9c50733ec11570ec)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:54.641852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ef553-3993-4d82-bfc9-49b597d257bc None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:54.645158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 162/332] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:05:54 2022] GET /identity/v3/users/d2640f0f0e6d418d9c50733ec11570ec => generated 293 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:05:54.654747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ef553-3993-4d82-bfc9-49b597d257bc None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:54.691388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 171/333] 10.222.0.64 () {58 vars in 1334 bytes} [Fri Sep 23 01:05:54 2022] PUT /identity/v3/system/users/aaa76e79ab1347a9834648c82a07abe1/roles/558d573b58fa4ba59d56e645fd5a0000 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:05:54.833683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:54.888827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ef553-3993-4d82-bfc9-49b597d257bc None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:05:54.962059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871e457-df6b-4fdd-ac2b-5d2234de3f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:05:54.963166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e457-df6b-4fdd-ac2b-5d2234de3f1a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:54.963712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e457-df6b-4fdd-ac2b-5d2234de3f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:54.964213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871e457-df6b-4fdd-ac2b-5d2234de3f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:54.965751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871e457-df6b-4fdd-ac2b-5d2234de3f1a None admin] RBAC: Adding query filter params (role.id=e5a65ae2d0884e419cdbc70c9223eac2, user.id=d2640f0f0e6d418d9c50733ec11570ec, scope.system=all) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:05:54.966366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871e457-df6b-4fdd-ac2b-5d2234de3f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:05:54.969196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871e457-df6b-4fdd-ac2b-5d2234de3f1a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:05:54.996827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 163/334] 10.222.0.64 () {56 vars in 1308 bytes} [Fri Sep 23 01:05:54 2022] GET /identity/v3/role_assignments?user.id=d2640f0f0e6d418d9c50733ec11570ec&scope.system=all&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 510 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:05:58.685608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82376fd-39fd-4c8b-bf85-f9ecf74fad08 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:58.687289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82376fd-39fd-4c8b-bf85-f9ecf74fad08 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:58.688223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82376fd-39fd-4c8b-bf85-f9ecf74fad08 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:58.691859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 172/335] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:05:58.704216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227cc19-1dbe-49a8-907c-7aa9f13e4bf8 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:58.705824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227cc19-1dbe-49a8-907c-7aa9f13e4bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:58.706573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227cc19-1dbe-49a8-907c-7aa9f13e4bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:58.747824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8227cc19-1dbe-49a8-907c-7aa9f13e4bf8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:58.750625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8227cc19-1dbe-49a8-907c-7aa9f13e4bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:58.933123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ed879-f88f-4d76-ad5c-a2ba90e3b064 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:58.934094 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ed879-f88f-4d76-ad5c-a2ba90e3b064 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:58.934780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ed879-f88f-4d76-ad5c-a2ba90e3b064 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:58.937751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 173/336] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:05:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:05:58.941592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 164/337] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:58.955689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00c39b-409c-415e-b0c7-f782decb9619 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:05:58.956571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00c39b-409c-415e-b0c7-f782decb9619 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:05:58.957219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00c39b-409c-415e-b0c7-f782decb9619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:05:59.003870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-bb00c39b-409c-415e-b0c7-f782decb9619 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:05:59.006560 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bb00c39b-409c-415e-b0c7-f782decb9619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:05:59.179242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 174/338] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:05:59.701963 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227cc19-1dbe-49a8-907c-7aa9f13e4bf8 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:05:59.870923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:05:59.966894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227cc19-1dbe-49a8-907c-7aa9f13e4bf8 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:00.065999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e69d-b269-4a1f-958a-f55b657a4cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:00.066968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e69d-b269-4a1f-958a-f55b657a4cfa None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:00.067384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e69d-b269-4a1f-958a-f55b657a4cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:00.067834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e69d-b269-4a1f-958a-f55b657a4cfa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:00.073196 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58961bed-8b63-4fde-96c8-6df52833750e None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:00.073932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58961bed-8b63-4fde-96c8-6df52833750e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:00.074180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58961bed-8b63-4fde-96c8-6df52833750e None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:00.077571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 175/339] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:00.094064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcd815-b308-4732-9c03-b023fa65a531 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:00.095042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcd815-b308-4732-9c03-b023fa65a531 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:00.095827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcd815-b308-4732-9c03-b023fa65a531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:00.100389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57e69d-b269-4a1f-958a-f55b657a4cfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-bf57e69d-b269-4a1f-958a-f55b657a4cfa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:00.111587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:00.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:00.114729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 165/340] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:05:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:00.124677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e69d-b269-4a1f-958a-f55b657a4cfa None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:00.149283 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-16fcd815-b308-4732-9c03-b023fa65a531 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:00.160219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-16fcd815-b308-4732-9c03-b023fa65a531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:00.420211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 176/341] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:00.421882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcd815-b308-4732-9c03-b023fa65a531 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:00.445518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:00.465489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e69d-b269-4a1f-958a-f55b657a4cfa None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:00.556226 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:00.601623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a414cb-6174-4e23-b0ff-6de650fa1246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:00.602692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a414cb-6174-4e23-b0ff-6de650fa1246 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:00.603169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a414cb-6174-4e23-b0ff-6de650fa1246 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:00.603707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a414cb-6174-4e23-b0ff-6de650fa1246 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:00.604997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a414cb-6174-4e23-b0ff-6de650fa1246 None admin] RBAC: Adding query filter params (name=member) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:00.605538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a414cb-6174-4e23-b0ff-6de650fa1246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:00.608301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a414cb-6174-4e23-b0ff-6de650fa1246 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:00.654644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 166/342] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:06:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 531 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:00.656740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc602ac4-47d5-4be4-aa95-a49e18b197c5 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:00.657182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc602ac4-47d5-4be4-aa95-a49e18b197c5 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:00.657613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc602ac4-47d5-4be4-aa95-a49e18b197c5 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:00.660738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 167/343] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:00.669517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc602ac4-47d5-4be4-aa95-a49e18b197c5 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:00.769072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcd815-b308-4732-9c03-b023fa65a531 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:00.916578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4093e-9ce0-4612-bf75-249a8882e904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:00.916578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4093e-9ce0-4612-bf75-249a8882e904 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:00.916968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4093e-9ce0-4612-bf75-249a8882e904 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:00.917294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4093e-9ce0-4612-bf75-249a8882e904 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:00.957206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4093e-9ce0-4612-bf75-249a8882e904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-3eb4093e-9ce0-4612-bf75-249a8882e904 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:00.964191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:00.965420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:00.966836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:00.970908 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 177/344] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:06:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 550 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:00.974080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87e6e4-743b-4eeb-afac-a45dae318ad2 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:00.974863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87e6e4-743b-4eeb-afac-a45dae318ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:00.978431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87e6e4-743b-4eeb-afac-a45dae318ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:00.986281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc602ac4-47d5-4be4-aa95-a49e18b197c5 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:01.030869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ba87e6e4-743b-4eeb-afac-a45dae318ad2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:01.032277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ba87e6e4-743b-4eeb-afac-a45dae318ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:01.079062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b37f-f584-492b-a7fb-dcf82f055424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:01.081037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b37f-f584-492b-a7fb-dcf82f055424 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:01.082316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b37f-f584-492b-a7fb-dcf82f055424 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:01.083000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b37f-f584-492b-a7fb-dcf82f055424 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:01.169754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:01.187880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 178/345] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:01.190288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87e6e4-743b-4eeb-afac-a45dae318ad2 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:01.208897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:01.235899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9b37f-f584-492b-a7fb-dcf82f055424 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:01.240401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb9b37f-f584-492b-a7fb-dcf82f055424 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:01.479587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87e6e4-743b-4eeb-afac-a45dae318ad2 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:01.554424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25018743-341e-4a14-ab8c-e1c0d86a4141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:01.556177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25018743-341e-4a14-ab8c-e1c0d86a4141 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:01.556999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25018743-341e-4a14-ab8c-e1c0d86a4141 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:01.557750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25018743-341e-4a14-ab8c-e1c0d86a4141 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:01.559385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25018743-341e-4a14-ab8c-e1c0d86a4141 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:01.560527 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25018743-341e-4a14-ab8c-e1c0d86a4141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:01.563514 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25018743-341e-4a14-ab8c-e1c0d86a4141 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:01.583027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 179/346] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:01.585602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25018743-341e-4a14-ab8c-e1c0d86a4141 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:01.611124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 168/347] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 943 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:01.613039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b37f-f584-492b-a7fb-dcf82f055424 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:01.723789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:01.837330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25018743-341e-4a14-ab8c-e1c0d86a4141 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:01.883593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:01.902962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b37f-f584-492b-a7fb-dcf82f055424 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:01.920383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d280c-0ac6-4f5e-bb01-7b20633c7d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:01.921460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d280c-0ac6-4f5e-bb01-7b20633c7d77 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:01.921933 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d280c-0ac6-4f5e-bb01-7b20633c7d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:01.922398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d280c-0ac6-4f5e-bb01-7b20633c7d77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:01.944336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d280c-0ac6-4f5e-bb01-7b20633c7d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-930d280c-0ac6-4f5e-bb01-7b20633c7d77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:01.954459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:01.956132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:01.957419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 180/348] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:01.958478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d280c-0ac6-4f5e-bb01-7b20633c7d77 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:01.982622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf342d-78c8-4ca3-9456-db21af1bc9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:01.983729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf342d-78c8-4ca3-9456-db21af1bc9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:01.984212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf342d-78c8-4ca3-9456-db21af1bc9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:01.985425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf342d-78c8-4ca3-9456-db21af1bc9a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:02.104542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf342d-78c8-4ca3-9456-db21af1bc9a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:02.106308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf342d-78c8-4ca3-9456-db21af1bc9a0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:02.165093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d280c-0ac6-4f5e-bb01-7b20633c7d77 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:02.200767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:02.233325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bccfdf-e53f-4643-b544-85846d365aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:02.234155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bccfdf-e53f-4643-b544-85846d365aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:02.234437 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bccfdf-e53f-4643-b544-85846d365aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:02.234715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bccfdf-e53f-4643-b544-85846d365aa2 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:02.236357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bccfdf-e53f-4643-b544-85846d365aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:02.239304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bccfdf-e53f-4643-b544-85846d365aa2 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:02.241309 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 181/349] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:01 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:02.242809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b3ac6-f507-4f8c-a886-817ca16c0137 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:02.243129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b3ac6-f507-4f8c-a886-817ca16c0137 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:02.243411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b3ac6-f507-4f8c-a886-817ca16c0137 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:02.245489 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 182/350] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:02.246508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b3ac6-f507-4f8c-a886-817ca16c0137 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:02.264015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:02.294338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 169/351] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:02.296031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b24bc-ce37-489a-8e38-c54f94de57ba None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:02.296501 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b24bc-ce37-489a-8e38-c54f94de57ba None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:02.296785 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b24bc-ce37-489a-8e38-c54f94de57ba None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:02.298273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 170/352] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:02.300241 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b24bc-ce37-489a-8e38-c54f94de57ba None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:02.444909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b3ac6-f507-4f8c-a886-817ca16c0137 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:02.499287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b24bc-ce37-489a-8e38-c54f94de57ba None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:02.518349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2696d-7e5c-4ea7-8121-11bb6c054437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:02.519097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2696d-7e5c-4ea7-8121-11bb6c054437 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:02.519407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2696d-7e5c-4ea7-8121-11bb6c054437 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:02.519720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2696d-7e5c-4ea7-8121-11bb6c054437 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:02.535394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2696d-7e5c-4ea7-8121-11bb6c054437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-6aa2696d-7e5c-4ea7-8121-11bb6c054437 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:02.540963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:02.542049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:02.542852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:02.545121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 183/353] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:02.546197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2696d-7e5c-4ea7-8121-11bb6c054437 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:02.573387 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c440ed3-1b6c-4913-ac20-1316ab31644e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:02.575197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c440ed3-1b6c-4913-ac20-1316ab31644e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:02.575765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c440ed3-1b6c-4913-ac20-1316ab31644e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:02.576488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c440ed3-1b6c-4913-ac20-1316ab31644e None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:02.577803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c440ed3-1b6c-4913-ac20-1316ab31644e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:02.578332 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c440ed3-1b6c-4913-ac20-1316ab31644e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:02.580731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c440ed3-1b6c-4913-ac20-1316ab31644e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:02.597159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 171/354] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:06:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:02.599254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc31c9a-6363-4bda-89d1-70e9af4966a0 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:02.599813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc31c9a-6363-4bda-89d1-70e9af4966a0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:02.600307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc31c9a-6363-4bda-89d1-70e9af4966a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:02.631101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-1fc31c9a-6363-4bda-89d1-70e9af4966a0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:02.633300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1fc31c9a-6363-4bda-89d1-70e9af4966a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:02.640609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:02.724825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:02.747864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2696d-7e5c-4ea7-8121-11bb6c054437 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:02.784660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 172/355] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:02.786556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc31c9a-6363-4bda-89d1-70e9af4966a0 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:02.823457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f979e8-17c7-4a14-ae77-9c91246b64b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:02.825116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f979e8-17c7-4a14-ae77-9c91246b64b5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:02.825602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f979e8-17c7-4a14-ae77-9c91246b64b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:02.826098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f979e8-17c7-4a14-ae77-9c91246b64b5 None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:02.841060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f979e8-17c7-4a14-ae77-9c91246b64b5 None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:02.843948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f979e8-17c7-4a14-ae77-9c91246b64b5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:02.848503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 184/356] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:02 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:02.851186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ab68f-ad9a-4502-82f8-a702b4aaaf18 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:02.851703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ab68f-ad9a-4502-82f8-a702b4aaaf18 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:02.852434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ab68f-ad9a-4502-82f8-a702b4aaaf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:02.887367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-950ab68f-ad9a-4502-82f8-a702b4aaaf18 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:02.890080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-950ab68f-ad9a-4502-82f8-a702b4aaaf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:03.027709 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc31c9a-6363-4bda-89d1-70e9af4966a0 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:03.044084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 185/357] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:03.045920 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ab68f-ad9a-4502-82f8-a702b4aaaf18 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:03.106880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00d8c7-bfb0-4085-8062-9538305dfd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:03.108192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00d8c7-bfb0-4085-8062-9538305dfd0b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:03.108793 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00d8c7-bfb0-4085-8062-9538305dfd0b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:03.109409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00d8c7-bfb0-4085-8062-9538305dfd0b None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:03.111740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00d8c7-bfb0-4085-8062-9538305dfd0b None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:03.115232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00d8c7-bfb0-4085-8062-9538305dfd0b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:03.119054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 173/358] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:02 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:03.120617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00d8c7-bfb0-4085-8062-9538305dfd0b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:03.135482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:03.217489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:03.279611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ab68f-ad9a-4502-82f8-a702b4aaaf18 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:03.346271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f00d70-e558-4ec4-b638-a11aa7e55195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:03.347306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f00d70-e558-4ec4-b638-a11aa7e55195 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:03.347472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00d8c7-bfb0-4085-8062-9538305dfd0b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:03.347795 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f00d70-e558-4ec4-b638-a11aa7e55195 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:03.348240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f00d70-e558-4ec4-b638-a11aa7e55195 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:03.349403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f00d70-e558-4ec4-b638-a11aa7e55195 None admin] RBAC: Adding query filter params (name=member) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:03.349902 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f00d70-e558-4ec4-b638-a11aa7e55195 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:03.352222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f00d70-e558-4ec4-b638-a11aa7e55195 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:03.367765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 186/359] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:06:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:03.372171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f00d70-e558-4ec4-b638-a11aa7e55195 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:03.423374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4fcf6-9c11-4230-9fec-ac4523f5f92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:03.424868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4fcf6-9c11-4230-9fec-ac4523f5f92c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:03.425566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4fcf6-9c11-4230-9fec-ac4523f5f92c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:03.426262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4fcf6-9c11-4230-9fec-ac4523f5f92c None admin] PATH_INFO: `/v3/users/aaa76e79ab1347a9834648c82a07abe1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:03.433442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4fcf6-9c11-4230-9fec-ac4523f5f92c None admin] RBAC: Authorizing `identity:get_user(user_id=aaa76e79ab1347a9834648c82a07abe1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:03.436724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4fcf6-9c11-4230-9fec-ac4523f5f92c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:03.440980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 174/360] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:06:03 2022] GET /identity/v3/users/aaa76e79ab1347a9834648c82a07abe1 => generated 293 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:03.442271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4fcf6-9c11-4230-9fec-ac4523f5f92c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:03.590804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f00d70-e558-4ec4-b638-a11aa7e55195 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:03.594146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:03.635098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3089c-1e41-456a-96ef-1ca037647b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:03.636007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3089c-1e41-456a-96ef-1ca037647b43 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:03.636412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3089c-1e41-456a-96ef-1ca037647b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:03.636779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d3089c-1e41-456a-96ef-1ca037647b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:03.637820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3089c-1e41-456a-96ef-1ca037647b43 None admin] RBAC: Adding query filter params (group.id=a0ade514dc644430b7241726dbed93e1, role.id=e5a65ae2d0884e419cdbc70c9223eac2, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:03.638229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3089c-1e41-456a-96ef-1ca037647b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:03.640165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d3089c-1e41-456a-96ef-1ca037647b43 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:03.648357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 187/361] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Sep 23 01:06:03 2022] GET /identity/v3/role_assignments?group.id=a0ade514dc644430b7241726dbed93e1&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 260 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:03.649538 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3089c-1e41-456a-96ef-1ca037647b43 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:03.657223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4fcf6-9c11-4230-9fec-ac4523f5f92c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:03.686935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:03.730218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53114e26-369a-4666-9c6a-82aad61d5f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:03.731423 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53114e26-369a-4666-9c6a-82aad61d5f07 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:03.731955 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53114e26-369a-4666-9c6a-82aad61d5f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:03.732449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53114e26-369a-4666-9c6a-82aad61d5f07 None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:03.734726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53114e26-369a-4666-9c6a-82aad61d5f07 None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:03.737822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53114e26-369a-4666-9c6a-82aad61d5f07 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:03.748365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 175/362] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:03 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:03.748365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53114e26-369a-4666-9c6a-82aad61d5f07 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:03.838576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d3089c-1e41-456a-96ef-1ca037647b43 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:03.923038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8352f-6a0f-4c49-8678-187d48b641fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:03.924314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8352f-6a0f-4c49-8678-187d48b641fc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:03.924888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8352f-6a0f-4c49-8678-187d48b641fc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:03.926382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8352f-6a0f-4c49-8678-187d48b641fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:03.945068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53114e26-369a-4666-9c6a-82aad61d5f07 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:03.948397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8352f-6a0f-4c49-8678-187d48b641fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-5cc8352f-6a0f-4c49-8678-187d48b641fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:03.954696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:03.956797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 188/363] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:06:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:03.957639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8352f-6a0f-4c49-8678-187d48b641fc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:04.005579 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:04.031904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665068f-a03f-4a56-a989-244bd47dd6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:04.033058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665068f-a03f-4a56-a989-244bd47dd6ac None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:04.033556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665068f-a03f-4a56-a989-244bd47dd6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:04.034042 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665068f-a03f-4a56-a989-244bd47dd6ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:04.125743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:04.158192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8352f-6a0f-4c49-8678-187d48b641fc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:04.171655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665068f-a03f-4a56-a989-244bd47dd6ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:04.176836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665068f-a03f-4a56-a989-244bd47dd6ac None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:04.213933 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5e53c-7fee-4c0d-bf31-22b1b268c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:04.214759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5e53c-7fee-4c0d-bf31-22b1b268c82c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:04.215182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5e53c-7fee-4c0d-bf31-22b1b268c82c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:04.215625 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5e53c-7fee-4c0d-bf31-22b1b268c82c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:04.216597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5e53c-7fee-4c0d-bf31-22b1b268c82c None admin] RBAC: Adding query filter params (role.id=558d573b58fa4ba59d56e645fd5a0000, user.id=aaa76e79ab1347a9834648c82a07abe1, scope.system=all) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:04.217066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5e53c-7fee-4c0d-bf31-22b1b268c82c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:04.219183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5e53c-7fee-4c0d-bf31-22b1b268c82c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:04.237399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 189/364] 10.222.0.64 () {56 vars in 1308 bytes} [Fri Sep 23 01:06:03 2022] GET /identity/v3/role_assignments?user.id=aaa76e79ab1347a9834648c82a07abe1&scope.system=all&role.id=558d573b58fa4ba59d56e645fd5a0000 => generated 510 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:04.238561 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5e53c-7fee-4c0d-bf31-22b1b268c82c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:04.406621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 176/365] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:04.409116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665068f-a03f-4a56-a989-244bd47dd6ac None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:04.420191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:04.440121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5e53c-7fee-4c0d-bf31-22b1b268c82c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:04.508024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b24f5-206d-4bd7-8b9a-9a12b1045291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:04.509280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b24f5-206d-4bd7-8b9a-9a12b1045291 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:04.509793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b24f5-206d-4bd7-8b9a-9a12b1045291 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:04.510409 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b24f5-206d-4bd7-8b9a-9a12b1045291 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:04.530617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b24f5-206d-4bd7-8b9a-9a12b1045291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-df6b24f5-206d-4bd7-8b9a-9a12b1045291 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:04.537004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:04.538718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:06:04.540585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:04.544610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 190/366] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:06:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:04.544956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b24f5-206d-4bd7-8b9a-9a12b1045291 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:04.603300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:04.656535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665068f-a03f-4a56-a989-244bd47dd6ac None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:04.726379 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f139106-17a3-46a9-8532-479b8047fec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:04.727320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f139106-17a3-46a9-8532-479b8047fec4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:04.727866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f139106-17a3-46a9-8532-479b8047fec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:04.728370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f139106-17a3-46a9-8532-479b8047fec4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:04.731596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f139106-17a3-46a9-8532-479b8047fec4 None admin] RBAC: Adding query filter params (group.id=a0ade514dc644430b7241726dbed93e1, role.id=b9c95d84eaca4444b2c7b8b5eed61a59, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:04.731596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f139106-17a3-46a9-8532-479b8047fec4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:04.733307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f139106-17a3-46a9-8532-479b8047fec4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:04.749744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 177/367] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Sep 23 01:06:04 2022] GET /identity/v3/role_assignments?group.id=a0ade514dc644430b7241726dbed93e1&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=b9c95d84eaca4444b2c7b8b5eed61a59 => generated 260 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:04.751954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f139106-17a3-46a9-8532-479b8047fec4 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:04.797355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b24f5-206d-4bd7-8b9a-9a12b1045291 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:04.887294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d504384-9caa-406e-b9c3-02c452569783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:04.888477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d504384-9caa-406e-b9c3-02c452569783 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:04.888954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d504384-9caa-406e-b9c3-02c452569783 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:04.889429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d504384-9caa-406e-b9c3-02c452569783 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:04.890713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d504384-9caa-406e-b9c3-02c452569783 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:04.891229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d504384-9caa-406e-b9c3-02c452569783 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:04.894581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d504384-9caa-406e-b9c3-02c452569783 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:04.914939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 191/368] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:04.916515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d504384-9caa-406e-b9c3-02c452569783 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:04.944442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:04.984841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f139106-17a3-46a9-8532-479b8047fec4 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:05.047163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a3019-0301-4c90-ac01-325015f2d42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:05.048783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a3019-0301-4c90-ac01-325015f2d42b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:05.049717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a3019-0301-4c90-ac01-325015f2d42b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:05.050404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a3019-0301-4c90-ac01-325015f2d42b None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:05.053585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a3019-0301-4c90-ac01-325015f2d42b None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:05.057091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a3019-0301-4c90-ac01-325015f2d42b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:05.059865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 178/369] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:04 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:05.064506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a3019-0301-4c90-ac01-325015f2d42b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:05.099189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:05.172415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d504384-9caa-406e-b9c3-02c452569783 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:05.255450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c45538-37b1-447c-9dcd-0d8b3c2f3371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:05.257794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c45538-37b1-447c-9dcd-0d8b3c2f3371 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:05.258602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c45538-37b1-447c-9dcd-0d8b3c2f3371 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:05.259342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c45538-37b1-447c-9dcd-0d8b3c2f3371 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:05.261097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c45538-37b1-447c-9dcd-0d8b3c2f3371 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:05.261961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c45538-37b1-447c-9dcd-0d8b3c2f3371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:05.265550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c45538-37b1-447c-9dcd-0d8b3c2f3371 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:05.287285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 192/370] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:05.289750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c45538-37b1-447c-9dcd-0d8b3c2f3371 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:05.306054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a3019-0301-4c90-ac01-325015f2d42b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:05.366169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecd142-d832-4048-b526-c81e7ce714c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:05.366906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecd142-d832-4048-b526-c81e7ce714c3 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:05.367187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecd142-d832-4048-b526-c81e7ce714c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:05.367462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecd142-d832-4048-b526-c81e7ce714c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:05.454926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:05.489783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecd142-d832-4048-b526-c81e7ce714c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:05.492059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cecd142-d832-4048-b526-c81e7ce714c3 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:05.523230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:05.529623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c45538-37b1-447c-9dcd-0d8b3c2f3371 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:05.598767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28801be-c399-437a-bce5-d27cb45c43b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:05.599645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28801be-c399-437a-bce5-d27cb45c43b9 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:05.599961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28801be-c399-437a-bce5-d27cb45c43b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:05.600244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28801be-c399-437a-bce5-d27cb45c43b9 None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:05.605085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28801be-c399-437a-bce5-d27cb45c43b9 None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:05.608429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28801be-c399-437a-bce5-d27cb45c43b9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:05.613028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 193/371] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:05 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:05.615425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28801be-c399-437a-bce5-d27cb45c43b9 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:05.702617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 179/372] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:05.704823 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecd142-d832-4048-b526-c81e7ce714c3 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:05.794096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28801be-c399-437a-bce5-d27cb45c43b9 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:05.865940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1efc86-da04-417e-a696-05559a64f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:05.867192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1efc86-da04-417e-a696-05559a64f97a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:05.867817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1efc86-da04-417e-a696-05559a64f97a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:05.868375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1efc86-da04-417e-a696-05559a64f97a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:05.886385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:05.949015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecd142-d832-4048-b526-c81e7ce714c3 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:05.979787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:06.016483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1efc86-da04-417e-a696-05559a64f97a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:06.020240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1efc86-da04-417e-a696-05559a64f97a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:06.024490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e08e5-d344-4c14-8934-630e648deb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:06.027042 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e08e5-d344-4c14-8934-630e648deb09 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:06.028033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e08e5-d344-4c14-8934-630e648deb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:06.029006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e08e5-d344-4c14-8934-630e648deb09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:06.030840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e08e5-d344-4c14-8934-630e648deb09 None admin] RBAC: Adding query filter params (group.id=a0ade514dc644430b7241726dbed93e1, role.id=e5a65ae2d0884e419cdbc70c9223eac2, scope.project.id=ac99f487657f41258cd0118165066bb1) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:06.031639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e08e5-d344-4c14-8934-630e648deb09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:06.035781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e08e5-d344-4c14-8934-630e648deb09 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:06.048908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 180/373] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Sep 23 01:06:05 2022] GET /identity/v3/role_assignments?group.id=a0ade514dc644430b7241726dbed93e1&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 260 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:06.051099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e08e5-d344-4c14-8934-630e648deb09 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:06.359319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 194/374] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:06.369809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1efc86-da04-417e-a696-05559a64f97a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:06.434424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e08e5-d344-4c14-8934-630e648deb09 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:06.497794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:06.510238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd26e9-4f26-4485-9c9a-49bcf1ae399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:06.511916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd26e9-4f26-4485-9c9a-49bcf1ae399a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:06.512672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd26e9-4f26-4485-9c9a-49bcf1ae399a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:06.516795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd26e9-4f26-4485-9c9a-49bcf1ae399a None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:06.516795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd26e9-4f26-4485-9c9a-49bcf1ae399a None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:06.520327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd26e9-4f26-4485-9c9a-49bcf1ae399a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:06.524516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 181/375] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:06 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:06.533838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd26e9-4f26-4485-9c9a-49bcf1ae399a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:06.654876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:06.676016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1efc86-da04-417e-a696-05559a64f97a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:06.790472 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e464bb7-8260-4735-bb38-2cbdefb79618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:06.791579 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e464bb7-8260-4735-bb38-2cbdefb79618 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:06.792067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e464bb7-8260-4735-bb38-2cbdefb79618 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:06.792538 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e464bb7-8260-4735-bb38-2cbdefb79618 None admin] PATH_INFO: `/v3/projects/9c4e8ab21bdd4e5ebc7ac91f1638bec3` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:06.800687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e464bb7-8260-4735-bb38-2cbdefb79618 None admin] RBAC: Authorizing `identity:get_project(project_id=9c4e8ab21bdd4e5ebc7ac91f1638bec3)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:06.803677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e464bb7-8260-4735-bb38-2cbdefb79618 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:06.807417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 195/376] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:06 2022] GET /identity/v3/projects/9c4e8ab21bdd4e5ebc7ac91f1638bec3 => generated 344 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:06.817921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e464bb7-8260-4735-bb38-2cbdefb79618 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:06.841217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd26e9-4f26-4485-9c9a-49bcf1ae399a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:06.951110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520122d3-59c2-4d8b-817f-f9108ac01a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:06.952224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520122d3-59c2-4d8b-817f-f9108ac01a26 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:06.952740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520122d3-59c2-4d8b-817f-f9108ac01a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:06.953189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520122d3-59c2-4d8b-817f-f9108ac01a26 None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:06.956086 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520122d3-59c2-4d8b-817f-f9108ac01a26 None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:06.958898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520122d3-59c2-4d8b-817f-f9108ac01a26 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:06.967600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 182/377] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:06 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:06.983769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520122d3-59c2-4d8b-817f-f9108ac01a26 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:07.143309 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e464bb7-8260-4735-bb38-2cbdefb79618 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:07.175752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:07.276584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4831f92-7427-46b1-9e60-fe5c49075912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:07.278070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4831f92-7427-46b1-9e60-fe5c49075912 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:07.278563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4831f92-7427-46b1-9e60-fe5c49075912 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:07.279055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4831f92-7427-46b1-9e60-fe5c49075912 None admin] PATH_INFO: `/v3/groups/408a1bec2bfc4869909cf1b991f17bfb` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:07.297979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4831f92-7427-46b1-9e60-fe5c49075912 None admin] RBAC: Authorizing `identity:get_group(group_id=408a1bec2bfc4869909cf1b991f17bfb)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:07.301349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4831f92-7427-46b1-9e60-fe5c49075912 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:07.304871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 196/378] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:06 2022] GET /identity/v3/groups/408a1bec2bfc4869909cf1b991f17bfb => generated 229 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:07.316262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4831f92-7427-46b1-9e60-fe5c49075912 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:07.332965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520122d3-59c2-4d8b-817f-f9108ac01a26 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:07.351608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:07.408278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55b5bf-093e-403d-9a96-7bf0758019c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:07.409867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55b5bf-093e-403d-9a96-7bf0758019c4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:07.410358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55b5bf-093e-403d-9a96-7bf0758019c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:07.410839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55b5bf-093e-403d-9a96-7bf0758019c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:07.412266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55b5bf-093e-403d-9a96-7bf0758019c4 None admin] RBAC: Adding query filter params (group.id=a0ade514dc644430b7241726dbed93e1, role.id=b9c95d84eaca4444b2c7b8b5eed61a59, scope.project.id=ac99f487657f41258cd0118165066bb1) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:07.412822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55b5bf-093e-403d-9a96-7bf0758019c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:07.415478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55b5bf-093e-403d-9a96-7bf0758019c4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:07.447505 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 183/379] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Sep 23 01:06:06 2022] GET /identity/v3/role_assignments?group.id=a0ade514dc644430b7241726dbed93e1&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=b9c95d84eaca4444b2c7b8b5eed61a59 => generated 260 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:07.537746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f22286-b6a5-4999-8e16-8c44146e7cca None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:07.538168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f22286-b6a5-4999-8e16-8c44146e7cca None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:07.538712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f22286-b6a5-4999-8e16-8c44146e7cca None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:07.541382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 184/380] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:06:07.551749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec973948-f3af-4fc1-9c5f-994f853bdffa None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:07.552275 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec973948-f3af-4fc1-9c5f-994f853bdffa None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:07.552558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec973948-f3af-4fc1-9c5f-994f853bdffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:07.605409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ec973948-f3af-4fc1-9c5f-994f853bdffa None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:07.608172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ec973948-f3af-4fc1-9c5f-994f853bdffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:07.610844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4831f92-7427-46b1-9e60-fe5c49075912 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:07.687262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd857f-3e0c-45a5-9bf4-f48de3ea1a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:07.688424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd857f-3e0c-45a5-9bf4-f48de3ea1a6a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:07.688925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd857f-3e0c-45a5-9bf4-f48de3ea1a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:07.689425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd857f-3e0c-45a5-9bf4-f48de3ea1a6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:07.692053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd857f-3e0c-45a5-9bf4-f48de3ea1a6a None admin] RBAC: Adding query filter params (group.id=408a1bec2bfc4869909cf1b991f17bfb, role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=9c4e8ab21bdd4e5ebc7ac91f1638bec3) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:07.692621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd857f-3e0c-45a5-9bf4-f48de3ea1a6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:07.699585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd857f-3e0c-45a5-9bf4-f48de3ea1a6a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:07.709568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 197/381] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Sep 23 01:06:07 2022] GET /identity/v3/role_assignments?group.id=408a1bec2bfc4869909cf1b991f17bfb&scope.project.id=9c4e8ab21bdd4e5ebc7ac91f1638bec3&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 260 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:07.840208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 185/382] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:08.538304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd857f-3e0c-45a5-9bf4-f48de3ea1a6a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:08.634940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:08.807888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd857f-3e0c-45a5-9bf4-f48de3ea1a6a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:08.891723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa66b4-f4c9-4316-935d-d9d6fc30bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:08.892389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa66b4-f4c9-4316-935d-d9d6fc30bdda None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:08.892696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa66b4-f4c9-4316-935d-d9d6fc30bdda None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:08.893518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa66b4-f4c9-4316-935d-d9d6fc30bdda None admin] PATH_INFO: `/v3/roles/member` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:08.921475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa66b4-f4c9-4316-935d-d9d6fc30bdda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-4cfa66b4-f4c9-4316-935d-d9d6fc30bdda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:08.935591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:08.937430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:08.938950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 198/383] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:08.953517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec973948-f3af-4fc1-9c5f-994f853bdffa None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:09.010306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65e62-da75-406d-9c01-408707e0322f None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:09.010760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65e62-da75-406d-9c01-408707e0322f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:09.011851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f65e62-da75-406d-9c01-408707e0322f None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:09.015224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 199/384] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:09.034355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad0bc3-f60a-4f54-b3e5-f5a71c6a16c0 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:09.035187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad0bc3-f60a-4f54-b3e5-f5a71c6a16c0 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:09.036076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad0bc3-f60a-4f54-b3e5-f5a71c6a16c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:09.068984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f0ad0bc3-f60a-4f54-b3e5-f5a71c6a16c0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:09.071672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f0ad0bc3-f60a-4f54-b3e5-f5a71c6a16c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:09.206248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 200/385] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:09.278495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec973948-f3af-4fc1-9c5f-994f853bdffa None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:09.298193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:09.368118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8222a44-5bc0-4791-96da-f13575b8d2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:09.369734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8222a44-5bc0-4791-96da-f13575b8d2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:09.370564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8222a44-5bc0-4791-96da-f13575b8d2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:09.371340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8222a44-5bc0-4791-96da-f13575b8d2c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:09.373051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8222a44-5bc0-4791-96da-f13575b8d2c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:09.373923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8222a44-5bc0-4791-96da-f13575b8d2c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:09.377084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8222a44-5bc0-4791-96da-f13575b8d2c6 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:09.401847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 186/386] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:06:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:09.411524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad0bc3-f60a-4f54-b3e5-f5a71c6a16c0 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:09.476907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:09.668100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad0bc3-f60a-4f54-b3e5-f5a71c6a16c0 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:09.756907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acf06b-9163-4d76-a524-001d4a50b6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:09.758055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acf06b-9163-4d76-a524-001d4a50b6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:09.758546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acf06b-9163-4d76-a524-001d4a50b6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:09.758901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acf06b-9163-4d76-a524-001d4a50b6d1 None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:09.761120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acf06b-9163-4d76-a524-001d4a50b6d1 None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:09.764167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acf06b-9163-4d76-a524-001d4a50b6d1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:09.768785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 201/387] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:09 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:09.777926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8222a44-5bc0-4791-96da-f13575b8d2c6 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:09.903528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acf06b-9163-4d76-a524-001d4a50b6d1 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:10.067619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8222a44-5bc0-4791-96da-f13575b8d2c6 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:10.113973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:10.133883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acf06b-9163-4d76-a524-001d4a50b6d1 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:10.196081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b0bee-77d9-4804-b358-63a55bde95a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:10.197233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b0bee-77d9-4804-b358-63a55bde95a2 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:10.197730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b0bee-77d9-4804-b358-63a55bde95a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:10.198191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b0bee-77d9-4804-b358-63a55bde95a2 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:10.209428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b0bee-77d9-4804-b358-63a55bde95a2 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:10.212719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b0bee-77d9-4804-b358-63a55bde95a2 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:10.220674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 187/388] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:09 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:10.227392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3faf0-7dbb-49c5-9939-7f9764f9201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:10.228589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3faf0-7dbb-49c5-9939-7f9764f9201a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:10.229011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b0bee-77d9-4804-b358-63a55bde95a2 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:10.229158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3faf0-7dbb-49c5-9939-7f9764f9201a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:10.229552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab3faf0-7dbb-49c5-9939-7f9764f9201a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:10.259318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab3faf0-7dbb-49c5-9939-7f9764f9201a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-eab3faf0-7dbb-49c5-9939-7f9764f9201a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:10.272519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:10.274442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:10.276064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:10.280843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 202/389] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:06:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:10.290530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3faf0-7dbb-49c5-9939-7f9764f9201a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:10.332534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:10.521205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b0bee-77d9-4804-b358-63a55bde95a2 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:10.630071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab3faf0-7dbb-49c5-9939-7f9764f9201a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:10.648212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c5501-fd1c-4742-bf1b-9a7fc22454b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:10.649397 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c5501-fd1c-4742-bf1b-9a7fc22454b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:10.649899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c5501-fd1c-4742-bf1b-9a7fc22454b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:10.650405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c5501-fd1c-4742-bf1b-9a7fc22454b9 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/groups/a0ade514dc644430b7241726dbed93e1/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:10.663932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c5501-fd1c-4742-bf1b-9a7fc22454b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfc85401ad649f0811b6b4ed33a5005, group_id=a0ade514dc644430b7241726dbed93e1, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:10.666910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c5501-fd1c-4742-bf1b-9a7fc22454b9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:10.727263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fa49d-009e-475b-a855-9b3d43cd07db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:10.728463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa49d-009e-475b-a855-9b3d43cd07db None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:10.728973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa49d-009e-475b-a855-9b3d43cd07db None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:10.729469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1fa49d-009e-475b-a855-9b3d43cd07db None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:10.730837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fa49d-009e-475b-a855-9b3d43cd07db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:10.731388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fa49d-009e-475b-a855-9b3d43cd07db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:10.734471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1fa49d-009e-475b-a855-9b3d43cd07db None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:10.750499 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 203/390] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:10.762138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fa49d-009e-475b-a855-9b3d43cd07db None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:10.767394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 188/391] 10.222.0.64 () {58 vars in 1442 bytes} [Fri Sep 23 01:06:10 2022] PUT /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/groups/a0ade514dc644430b7241726dbed93e1/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:06:10.772773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638fae9-ee64-4910-a3c4-49656300a667 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:10.773384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638fae9-ee64-4910-a3c4-49656300a667 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:10.773891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638fae9-ee64-4910-a3c4-49656300a667 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:10.777189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 189/392] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:10.803133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:10.807663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f632fd7-a1a5-490f-ad9c-48dfb725fd2e None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:10.808060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f632fd7-a1a5-490f-ad9c-48dfb725fd2e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:10.808365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f632fd7-a1a5-490f-ad9c-48dfb725fd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:10.887054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-4f632fd7-a1a5-490f-ad9c-48dfb725fd2e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:10.891956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4f632fd7-a1a5-490f-ad9c-48dfb725fd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:11.007113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1fa49d-009e-475b-a855-9b3d43cd07db None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:11.072276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e56aaf-8214-4679-a1ff-7490a93ad56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:11.073412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e56aaf-8214-4679-a1ff-7490a93ad56f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:11.073931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e56aaf-8214-4679-a1ff-7490a93ad56f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:11.074424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e56aaf-8214-4679-a1ff-7490a93ad56f None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:11.078165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e56aaf-8214-4679-a1ff-7490a93ad56f None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:11.083267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e56aaf-8214-4679-a1ff-7490a93ad56f None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:11.085691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 190/393] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:11.086822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 204/394] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:10 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:11.094730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f632fd7-a1a5-490f-ad9c-48dfb725fd2e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:11.135724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:11.318188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f632fd7-a1a5-490f-ad9c-48dfb725fd2e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:11.388914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91ccee-2e7c-42bc-b0cc-d43bf91a854b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:11.389953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91ccee-2e7c-42bc-b0cc-d43bf91a854b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:11.390420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91ccee-2e7c-42bc-b0cc-d43bf91a854b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:11.390898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91ccee-2e7c-42bc-b0cc-d43bf91a854b None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:11.393670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91ccee-2e7c-42bc-b0cc-d43bf91a854b None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:11.396623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91ccee-2e7c-42bc-b0cc-d43bf91a854b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:11.400398 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 191/395] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:11 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:11.410353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e56aaf-8214-4679-a1ff-7490a93ad56f None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:11.579924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc05e3e-1722-4712-bb45-9a34d7c8c53b None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:11.580409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc05e3e-1722-4712-bb45-9a34d7c8c53b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:11.580791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc05e3e-1722-4712-bb45-9a34d7c8c53b None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:11.583888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 192/396] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:11.601316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87fbbd-accf-493e-acb0-76781a956eb4 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:11.601762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87fbbd-accf-493e-acb0-76781a956eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:11.602165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87fbbd-accf-493e-acb0-76781a956eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:11.657394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:11.663301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8f87fbbd-accf-493e-acb0-76781a956eb4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:11.665535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8f87fbbd-accf-493e-acb0-76781a956eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:11.666482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e56aaf-8214-4679-a1ff-7490a93ad56f None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:11.741157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a94563-4035-4f4e-b6eb-f02f65078359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:11.742288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94563-4035-4f4e-b6eb-f02f65078359 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:11.742780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94563-4035-4f4e-b6eb-f02f65078359 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:11.743274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a94563-4035-4f4e-b6eb-f02f65078359 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/groups/a0ade514dc644430b7241726dbed93e1/roles/b9c95d84eaca4444b2c7b8b5eed61a59` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:11.747298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a94563-4035-4f4e-b6eb-f02f65078359 None admin] RBAC: Authorizing `identity:create_grant(project_id=5dfc85401ad649f0811b6b4ed33a5005, group_id=a0ade514dc644430b7241726dbed93e1, role_id=b9c95d84eaca4444b2c7b8b5eed61a59)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:11.750565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a94563-4035-4f4e-b6eb-f02f65078359 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:11.803058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 193/397] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:06:11.804435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f8a1a-3778-4d91-bf39-21822a1c9545 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:11.804726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f8a1a-3778-4d91-bf39-21822a1c9545 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:11.805006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697f8a1a-3778-4d91-bf39-21822a1c9545 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:11.813811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 194/398] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:11.813811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f8a1a-3778-4d91-bf39-21822a1c9545 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:11.842641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 205/399] 10.222.0.64 () {58 vars in 1442 bytes} [Fri Sep 23 01:06:11 2022] PUT /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005/groups/a0ade514dc644430b7241726dbed93e1/roles/b9c95d84eaca4444b2c7b8b5eed61a59 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:06:11.843465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a76921-62cc-4b5c-a3ad-8a5caaf3d089 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:11.843998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a76921-62cc-4b5c-a3ad-8a5caaf3d089 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:11.844475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a76921-62cc-4b5c-a3ad-8a5caaf3d089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:11.871911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c7a76921-62cc-4b5c-a3ad-8a5caaf3d089 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:11.873933 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c7a76921-62cc-4b5c-a3ad-8a5caaf3d089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:11.888939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:12.038066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 206/400] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:12.099974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697f8a1a-3778-4d91-bf39-21822a1c9545 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:12.182758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375cce1-6912-4507-a394-7ead4b2a5960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:12.184057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375cce1-6912-4507-a394-7ead4b2a5960 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:12.184559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375cce1-6912-4507-a394-7ead4b2a5960 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:12.185056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375cce1-6912-4507-a394-7ead4b2a5960 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:12.206500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375cce1-6912-4507-a394-7ead4b2a5960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-2375cce1-6912-4507-a394-7ead4b2a5960 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:12.216345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:12.217310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:12.218115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:12.218745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 195/401] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:12.227616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a76921-62cc-4b5c-a3ad-8a5caaf3d089 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:12.380520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:12.442926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375cce1-6912-4507-a394-7ead4b2a5960 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:12.443309 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a76921-62cc-4b5c-a3ad-8a5caaf3d089 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:12.538826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f1073-22c1-44a0-8354-60b1061cb496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:12.539968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f1073-22c1-44a0-8354-60b1061cb496 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:12.540523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f1073-22c1-44a0-8354-60b1061cb496 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:12.541144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f1073-22c1-44a0-8354-60b1061cb496 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:12.542088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f1073-22c1-44a0-8354-60b1061cb496 None admin] RBAC: Adding query filter params (name=member) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:12.542612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f1073-22c1-44a0-8354-60b1061cb496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:12.544357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f1073-22c1-44a0-8354-60b1061cb496 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:12.554990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 207/402] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:06:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:12.561268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f1073-22c1-44a0-8354-60b1061cb496 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:12.695869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:12.731875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f1073-22c1-44a0-8354-60b1061cb496 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:12.734965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375cce1-6912-4507-a394-7ead4b2a5960 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:12.795146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968de84-d987-4d04-be39-eb8d885fa9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:12.796006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968de84-d987-4d04-be39-eb8d885fa9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:12.796291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968de84-d987-4d04-be39-eb8d885fa9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:12.796581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1004d82-a62d-457e-83e3-947899f19a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:12.796837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968de84-d987-4d04-be39-eb8d885fa9d1 None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:12.797662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1004d82-a62d-457e-83e3-947899f19a83 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:12.798137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1004d82-a62d-457e-83e3-947899f19a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:12.798639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1004d82-a62d-457e-83e3-947899f19a83 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:12.798912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968de84-d987-4d04-be39-eb8d885fa9d1 None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:12.802691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968de84-d987-4d04-be39-eb8d885fa9d1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:12.811087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 208/403] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:12 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:12.812436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968de84-d987-4d04-be39-eb8d885fa9d1 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:12.820611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:12.821695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1004d82-a62d-457e-83e3-947899f19a83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-c1004d82-a62d-457e-83e3-947899f19a83 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:12.835628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:12.839111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:12.840104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:12.842813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 196/404] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:06:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:06:12.844092 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1004d82-a62d-457e-83e3-947899f19a83 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:12.994435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1004d82-a62d-457e-83e3-947899f19a83 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:12.994937 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968de84-d987-4d04-be39-eb8d885fa9d1 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:13.044675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f683d0-f928-4e53-9888-fc105e8736c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:13.045902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f683d0-f928-4e53-9888-fc105e8736c1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:13.046433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f683d0-f928-4e53-9888-fc105e8736c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:13.046913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f683d0-f928-4e53-9888-fc105e8736c1 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:13.048984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f683d0-f928-4e53-9888-fc105e8736c1 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:13.051911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f683d0-f928-4e53-9888-fc105e8736c1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:13.055137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 197/405] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:12 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:13.056725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f683d0-f928-4e53-9888-fc105e8736c1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:13.060497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce11bb-630e-4263-85c6-79ad4cb7df97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:13.063663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce11bb-630e-4263-85c6-79ad4cb7df97 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:13.064158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce11bb-630e-4263-85c6-79ad4cb7df97 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:13.064706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce11bb-630e-4263-85c6-79ad4cb7df97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:13.078330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce11bb-630e-4263-85c6-79ad4cb7df97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:13.081476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-27ce11bb-630e-4263-85c6-79ad4cb7df97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:13.086608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:13.088443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:06:13.089790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:13.091242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 209/406] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:06:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:13.092341 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce11bb-630e-4263-85c6-79ad4cb7df97 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:13.184076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:13.210684 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f683d0-f928-4e53-9888-fc105e8736c1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:13.237336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce11bb-630e-4263-85c6-79ad4cb7df97 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:13.259128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c777bd-e57a-4ad3-aad3-d3ea9e5ddb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:13.260564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c777bd-e57a-4ad3-aad3-d3ea9e5ddb8b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:13.261263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c777bd-e57a-4ad3-aad3-d3ea9e5ddb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:13.261862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c777bd-e57a-4ad3-aad3-d3ea9e5ddb8b None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:13.263406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c777bd-e57a-4ad3-aad3-d3ea9e5ddb8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:13.264214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c777bd-e57a-4ad3-aad3-d3ea9e5ddb8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:13.267254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c777bd-e57a-4ad3-aad3-d3ea9e5ddb8b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:13.284054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 198/407] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:13.285901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c777bd-e57a-4ad3-aad3-d3ea9e5ddb8b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:13.309113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa9273-feb9-4177-b85b-94b12b5adf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:13.311719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa9273-feb9-4177-b85b-94b12b5adf12 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:13.312490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa9273-feb9-4177-b85b-94b12b5adf12 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:13.313241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa9273-feb9-4177-b85b-94b12b5adf12 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1/groups/a0ade514dc644430b7241726dbed93e1/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:13.320129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa9273-feb9-4177-b85b-94b12b5adf12 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac99f487657f41258cd0118165066bb1, group_id=a0ade514dc644430b7241726dbed93e1, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:13.322884 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa9273-feb9-4177-b85b-94b12b5adf12 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:13.424828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:13.475307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c777bd-e57a-4ad3-aad3-d3ea9e5ddb8b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:13.516238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 210/408] 10.222.0.64 () {58 vars in 1442 bytes} [Fri Sep 23 01:06:13 2022] PUT /identity/v3/projects/ac99f487657f41258cd0118165066bb1/groups/a0ade514dc644430b7241726dbed93e1/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:06:13.524088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa9273-feb9-4177-b85b-94b12b5adf12 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:13.527358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd90f65-b76a-4768-973b-08d6550d577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:13.531960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd90f65-b76a-4768-973b-08d6550d577d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:13.531960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd90f65-b76a-4768-973b-08d6550d577d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:13.531960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd90f65-b76a-4768-973b-08d6550d577d None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:13.531960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd90f65-b76a-4768-973b-08d6550d577d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:13.531960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd90f65-b76a-4768-973b-08d6550d577d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:13.531960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd90f65-b76a-4768-973b-08d6550d577d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:13.541694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 199/409] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:13.548422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd90f65-b76a-4768-973b-08d6550d577d None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:13.755022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa9273-feb9-4177-b85b-94b12b5adf12 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:13.824365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb36267-564a-47ef-a4b8-73489b8328eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:13.828416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb36267-564a-47ef-a4b8-73489b8328eb None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:13.829041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb36267-564a-47ef-a4b8-73489b8328eb None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:13.829545 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:13.830764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb36267-564a-47ef-a4b8-73489b8328eb None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:13.832246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb36267-564a-47ef-a4b8-73489b8328eb None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:13.834268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb36267-564a-47ef-a4b8-73489b8328eb None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:13.837577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 211/410] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:13 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:13.845197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb36267-564a-47ef-a4b8-73489b8328eb None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:13.851050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd90f65-b76a-4768-973b-08d6550d577d None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:13.910547 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ed580-81b1-459e-8313-a924c3c89115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:13.910547 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ed580-81b1-459e-8313-a924c3c89115 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:13.910908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ed580-81b1-459e-8313-a924c3c89115 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:13.911409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ed580-81b1-459e-8313-a924c3c89115 None admin] PATH_INFO: `/v3/groups/408a1bec2bfc4869909cf1b991f17bfb` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:13.912890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ed580-81b1-459e-8313-a924c3c89115 None admin] RBAC: Authorizing `identity:get_group(group_id=408a1bec2bfc4869909cf1b991f17bfb)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:13.914510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ed580-81b1-459e-8313-a924c3c89115 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:13.921473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 200/411] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:13 2022] GET /identity/v3/groups/408a1bec2bfc4869909cf1b991f17bfb => generated 229 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:13.923197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ed580-81b1-459e-8313-a924c3c89115 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:13.942188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:14.037812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb36267-564a-47ef-a4b8-73489b8328eb None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:14.152664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ed580-81b1-459e-8313-a924c3c89115 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:14.163448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765af6d-5845-4e94-bf9d-4d397d831f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:14.164186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765af6d-5845-4e94-bf9d-4d397d831f28 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:14.164473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765af6d-5845-4e94-bf9d-4d397d831f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:14.164756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765af6d-5845-4e94-bf9d-4d397d831f28 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:14.168547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765af6d-5845-4e94-bf9d-4d397d831f28 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:14.170103 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765af6d-5845-4e94-bf9d-4d397d831f28 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:14.179149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 212/412] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:13 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:14.185466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765af6d-5845-4e94-bf9d-4d397d831f28 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:14.237297 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749c8cc-872e-43bc-adae-7a5ef428ca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:14.238404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749c8cc-872e-43bc-adae-7a5ef428ca7c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:14.238893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749c8cc-872e-43bc-adae-7a5ef428ca7c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:14.239378 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749c8cc-872e-43bc-adae-7a5ef428ca7c None admin] PATH_INFO: `/v3/projects/9c4e8ab21bdd4e5ebc7ac91f1638bec3` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:14.245003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749c8cc-872e-43bc-adae-7a5ef428ca7c None admin] RBAC: Authorizing `identity:get_project(project_id=9c4e8ab21bdd4e5ebc7ac91f1638bec3)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:14.247075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e749c8cc-872e-43bc-adae-7a5ef428ca7c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:14.249265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 201/413] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:13 2022] GET /identity/v3/projects/9c4e8ab21bdd4e5ebc7ac91f1638bec3 => generated 344 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:14.257110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749c8cc-872e-43bc-adae-7a5ef428ca7c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:14.287966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:14.343646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765af6d-5845-4e94-bf9d-4d397d831f28 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:14.387246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56438b-61a7-4859-aa58-7bff013e2981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:14.388212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56438b-61a7-4859-aa58-7bff013e2981 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:14.388701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56438b-61a7-4859-aa58-7bff013e2981 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:14.388977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56438b-61a7-4859-aa58-7bff013e2981 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1/groups/a0ade514dc644430b7241726dbed93e1/roles/b9c95d84eaca4444b2c7b8b5eed61a59` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:14.391223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56438b-61a7-4859-aa58-7bff013e2981 None admin] RBAC: Authorizing `identity:create_grant(project_id=ac99f487657f41258cd0118165066bb1, group_id=a0ade514dc644430b7241726dbed93e1, role_id=b9c95d84eaca4444b2c7b8b5eed61a59)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:14.392800 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a56438b-61a7-4859-aa58-7bff013e2981 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:14.401546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:14.442525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e749c8cc-872e-43bc-adae-7a5ef428ca7c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:14.476700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 213/414] 10.222.0.64 () {58 vars in 1442 bytes} [Fri Sep 23 01:06:14 2022] PUT /identity/v3/projects/ac99f487657f41258cd0118165066bb1/groups/a0ade514dc644430b7241726dbed93e1/roles/b9c95d84eaca4444b2c7b8b5eed61a59 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:06:14.482679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1d584-8d50-4cab-86c8-435c7e7638be None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:14.482679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1d584-8d50-4cab-86c8-435c7e7638be None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:14.482679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1d584-8d50-4cab-86c8-435c7e7638be None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:14.484128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 214/415] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:14.495502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8cf26-5f19-4e2b-b4dd-7d55a0997991 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:14.496820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8cf26-5f19-4e2b-b4dd-7d55a0997991 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:14.497815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8cf26-5f19-4e2b-b4dd-7d55a0997991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:14.501130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba74f2-d245-447b-9fac-9db8cef1a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:14.504507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba74f2-d245-447b-9fac-9db8cef1a5f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:14.505147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba74f2-d245-447b-9fac-9db8cef1a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:14.505675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba74f2-d245-447b-9fac-9db8cef1a5f9 None admin] PATH_INFO: `/v3/projects/9c4e8ab21bdd4e5ebc7ac91f1638bec3/groups/408a1bec2bfc4869909cf1b991f17bfb/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:14.511743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba74f2-d245-447b-9fac-9db8cef1a5f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c4e8ab21bdd4e5ebc7ac91f1638bec3, group_id=408a1bec2bfc4869909cf1b991f17bfb, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:14.517139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba74f2-d245-447b-9fac-9db8cef1a5f9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:14.541370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-28a8cf26-5f19-4e2b-b4dd-7d55a0997991 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:14.547874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-28a8cf26-5f19-4e2b-b4dd-7d55a0997991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:14.626576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 202/416] 10.222.0.64 () {58 vars in 1442 bytes} [Fri Sep 23 01:06:14 2022] PUT /identity/v3/projects/9c4e8ab21bdd4e5ebc7ac91f1638bec3/groups/408a1bec2bfc4869909cf1b991f17bfb/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:06:14.638595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 215/417] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:15.037308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba74f2-d245-447b-9fac-9db8cef1a5f9 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:15.249855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87869e9e-3b04-46c0-b9ef-c1049c2aadb7 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:15.250608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87869e9e-3b04-46c0-b9ef-c1049c2aadb7 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:15.250608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87869e9e-3b04-46c0-b9ef-c1049c2aadb7 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:15.250851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba74f2-d245-447b-9fac-9db8cef1a5f9 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:15.254332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 216/418] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:06:15.259008 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d57c-bab2-47a5-8999-6f9c188d150a None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:15.259346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d57c-bab2-47a5-8999-6f9c188d150a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:15.259743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884d57c-bab2-47a5-8999-6f9c188d150a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:15.285482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6884d57c-bab2-47a5-8999-6f9c188d150a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:15.286745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6884d57c-bab2-47a5-8999-6f9c188d150a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:15.344456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:15.352134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6f06e-4f13-4136-8cf3-6b640695e785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:15.352821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6f06e-4f13-4136-8cf3-6b640695e785 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:15.354531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6f06e-4f13-4136-8cf3-6b640695e785 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:15.354864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6f06e-4f13-4136-8cf3-6b640695e785 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:15.382705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e6f06e-4f13-4136-8cf3-6b640695e785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:15.394143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-76e6f06e-4f13-4136-8cf3-6b640695e785 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:15.404473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:15.416163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:15.417033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 203/419] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:15.417687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6f06e-4f13-4136-8cf3-6b640695e785 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:15.442055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 217/420] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:06:15.675118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6f06e-4f13-4136-8cf3-6b640695e785 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:15.745356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d81f7-85e7-4cba-9156-3e6ba7402506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:15.746345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d81f7-85e7-4cba-9156-3e6ba7402506 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:15.746821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d81f7-85e7-4cba-9156-3e6ba7402506 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:15.747311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d81f7-85e7-4cba-9156-3e6ba7402506 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:15.748647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d81f7-85e7-4cba-9156-3e6ba7402506 None admin] RBAC: Adding query filter params (name=member) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:15.749199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d81f7-85e7-4cba-9156-3e6ba7402506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:15.752041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d81f7-85e7-4cba-9156-3e6ba7402506 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:15.770249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 204/421] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:06:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:15.779319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884d57c-bab2-47a5-8999-6f9c188d150a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:15.815847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d81f7-85e7-4cba-9156-3e6ba7402506 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:15.964242 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:16.011169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884d57c-bab2-47a5-8999-6f9c188d150a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:16.057662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f7f2e-f589-4ed9-be0d-a16ea111b818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:16.058760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f7f2e-f589-4ed9-be0d-a16ea111b818 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:16.059461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f7f2e-f589-4ed9-be0d-a16ea111b818 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:16.060154 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3f7f2e-f589-4ed9-be0d-a16ea111b818 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:16.090179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d81f7-85e7-4cba-9156-3e6ba7402506 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:16.132643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:16.163882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f7f2e-f589-4ed9-be0d-a16ea111b818 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:16.165961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3f7f2e-f589-4ed9-be0d-a16ea111b818 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:16.238888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36f6ca-9ab9-4556-986c-0a61af785f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:16.239908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36f6ca-9ab9-4556-986c-0a61af785f11 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:16.240406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36f6ca-9ab9-4556-986c-0a61af785f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:16.240902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36f6ca-9ab9-4556-986c-0a61af785f11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:16.286835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36f6ca-9ab9-4556-986c-0a61af785f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-1c36f6ca-9ab9-4556-986c-0a61af785f11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:16.299799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:16.300875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:16.301673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:16.302277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 205/422] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:06:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 490 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:16.314699 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36f6ca-9ab9-4556-986c-0a61af785f11 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:16.441942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 218/423] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:16.449086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f7f2e-f589-4ed9-be0d-a16ea111b818 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:16.544886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:16.573039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3f7f2e-f589-4ed9-be0d-a16ea111b818 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:16.616904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711ef03-9170-42d3-b252-2935430aa246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:16.618683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711ef03-9170-42d3-b252-2935430aa246 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:16.618683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711ef03-9170-42d3-b252-2935430aa246 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:16.618683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711ef03-9170-42d3-b252-2935430aa246 None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:16.619779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711ef03-9170-42d3-b252-2935430aa246 None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:16.621383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711ef03-9170-42d3-b252-2935430aa246 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:16.623641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 219/424] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:16 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:16.629694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711ef03-9170-42d3-b252-2935430aa246 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:16.679026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36f6ca-9ab9-4556-986c-0a61af785f11 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:16.715194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:16.765473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711ef03-9170-42d3-b252-2935430aa246 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:16.796574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f35d11-7456-4d4d-9136-a0e0b70b05cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:16.797156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35d11-7456-4d4d-9136-a0e0b70b05cc None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:16.797616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35d11-7456-4d4d-9136-a0e0b70b05cc None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:16.797974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f35d11-7456-4d4d-9136-a0e0b70b05cc None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:16.798753 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f35d11-7456-4d4d-9136-a0e0b70b05cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:16.799177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f35d11-7456-4d4d-9136-a0e0b70b05cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:16.800672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f35d11-7456-4d4d-9136-a0e0b70b05cc None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:16.811390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3e208-a30e-4747-9433-b2001b729f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:16.811390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e208-a30e-4747-9433-b2001b729f18 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:16.811390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e208-a30e-4747-9433-b2001b729f18 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:16.811390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e208-a30e-4747-9433-b2001b729f18 None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:16.812070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3e208-a30e-4747-9433-b2001b729f18 None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:16.812198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3e208-a30e-4747-9433-b2001b729f18 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:16.816733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 220/425] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:16 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:16.821397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3e208-a30e-4747-9433-b2001b729f18 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:16.841885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 206/426] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 529 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:16.844013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:16.852804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f35d11-7456-4d4d-9136-a0e0b70b05cc None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:17.120950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3e208-a30e-4747-9433-b2001b729f18 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:17.171907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ced2de-06be-4e7c-8ce6-d4b0192dc17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:17.173318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced2de-06be-4e7c-8ce6-d4b0192dc17a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:17.174483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced2de-06be-4e7c-8ce6-d4b0192dc17a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:17.174965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ced2de-06be-4e7c-8ce6-d4b0192dc17a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:17.176095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ced2de-06be-4e7c-8ce6-d4b0192dc17a None admin] RBAC: Adding query filter params (group.id=a0ade514dc644430b7241726dbed93e1, role.id=e5a65ae2d0884e419cdbc70c9223eac2, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:17.176743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ced2de-06be-4e7c-8ce6-d4b0192dc17a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:17.178316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f35d11-7456-4d4d-9136-a0e0b70b05cc None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:17.179206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ced2de-06be-4e7c-8ce6-d4b0192dc17a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:17.187214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 221/427] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Sep 23 01:06:16 2022] GET /identity/v3/role_assignments?group.id=a0ade514dc644430b7241726dbed93e1&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 611 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:17.189242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eef54-3f78-421e-967b-76def0a3b968 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:17.190428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eef54-3f78-421e-967b-76def0a3b968 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:17.190961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2eef54-3f78-421e-967b-76def0a3b968 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:17.192794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 222/428] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:17.207994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99df96-4f21-4f70-b113-73e56d6ff999 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:17.208737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99df96-4f21-4f70-b113-73e56d6ff999 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:17.209507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99df96-4f21-4f70-b113-73e56d6ff999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:17.249411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-0f99df96-4f21-4f70-b113-73e56d6ff999 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:17.253939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0f99df96-4f21-4f70-b113-73e56d6ff999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:17.280806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de87463-247c-41b6-bfee-a83b26ca92a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:17.281926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de87463-247c-41b6-bfee-a83b26ca92a9 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:17.282437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de87463-247c-41b6-bfee-a83b26ca92a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:17.282951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de87463-247c-41b6-bfee-a83b26ca92a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:17.431510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:17.445535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 223/429] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:06:17.496241 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de87463-247c-41b6-bfee-a83b26ca92a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:17.499102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de87463-247c-41b6-bfee-a83b26ca92a9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:17.710110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 207/430] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 858 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:17.716462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99df96-4f21-4f70-b113-73e56d6ff999 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:17.732310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:17.845957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de87463-247c-41b6-bfee-a83b26ca92a9 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:17.897102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99df96-4f21-4f70-b113-73e56d6ff999 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:17.949707 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc68dd-b577-48da-9b37-5ae5f5c20f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:17.950899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc68dd-b577-48da-9b37-5ae5f5c20f0d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:17.951398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc68dd-b577-48da-9b37-5ae5f5c20f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:17.951910 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc68dd-b577-48da-9b37-5ae5f5c20f0d None admin] PATH_INFO: `/v3/projects/5dfc85401ad649f0811b6b4ed33a5005` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:17.953504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc68dd-b577-48da-9b37-5ae5f5c20f0d None admin] RBAC: Authorizing `identity:get_project(project_id=5dfc85401ad649f0811b6b4ed33a5005)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:17.956023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc68dd-b577-48da-9b37-5ae5f5c20f0d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:17.959153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 224/431] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:17 2022] GET /identity/v3/projects/5dfc85401ad649f0811b6b4ed33a5005 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:17.961230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d06b60-d6fa-4546-876a-b831db0dfc0f None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:17.961801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d06b60-d6fa-4546-876a-b831db0dfc0f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:17.962276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d06b60-d6fa-4546-876a-b831db0dfc0f None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:17.968513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 225/432] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:17.969495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d06b60-d6fa-4546-876a-b831db0dfc0f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:18.015697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:18.070696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de87463-247c-41b6-bfee-a83b26ca92a9 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:18.110531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:18.139529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e69f26-a293-4cdc-971d-465a29e9e0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:18.140851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e69f26-a293-4cdc-971d-465a29e9e0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:18.141320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e69f26-a293-4cdc-971d-465a29e9e0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:18.141803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e69f26-a293-4cdc-971d-465a29e9e0d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:18.157347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d06b60-d6fa-4546-876a-b831db0dfc0f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:18.161288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e69f26-a293-4cdc-971d-465a29e9e0d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-28e69f26-a293-4cdc-971d-465a29e9e0d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:18.167538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:18.169575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 208/433] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:18.171411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6680d-5978-4ec7-ba24-d1a9a2896c5f None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:18.174260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6680d-5978-4ec7-ba24-d1a9a2896c5f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:18.174260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6680d-5978-4ec7-ba24-d1a9a2896c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:18.191450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-23a6680d-5978-4ec7-ba24-d1a9a2896c5f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:18.192779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-23a6680d-5978-4ec7-ba24-d1a9a2896c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:18.215602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd495c6c-9a44-4581-a549-00d9bed60009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:18.216352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd495c6c-9a44-4581-a549-00d9bed60009 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:18.216734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd495c6c-9a44-4581-a549-00d9bed60009 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:18.217012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd495c6c-9a44-4581-a549-00d9bed60009 None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:18.218672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd495c6c-9a44-4581-a549-00d9bed60009 None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:18.222076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd495c6c-9a44-4581-a549-00d9bed60009 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:18.227806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 226/434] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:17 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:18.229736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd495c6c-9a44-4581-a549-00d9bed60009 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:18.317500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 209/435] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:18.319235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6680d-5978-4ec7-ba24-d1a9a2896c5f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:18.405731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:18.429372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd495c6c-9a44-4581-a549-00d9bed60009 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:18.467941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6680d-5978-4ec7-ba24-d1a9a2896c5f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:18.487640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8e26-2d98-428d-856a-f3009fc338f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:18.488455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8e26-2d98-428d-856a-f3009fc338f2 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:18.488623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8e26-2d98-428d-856a-f3009fc338f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:18.489113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8e26-2d98-428d-856a-f3009fc338f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:18.489994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd8e26-2d98-428d-856a-f3009fc338f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:18.490481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd8e26-2d98-428d-856a-f3009fc338f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:18.493515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd8e26-2d98-428d-856a-f3009fc338f2 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:18.514143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 227/436] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:06:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:18.517048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8e26-2d98-428d-856a-f3009fc338f2 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:18.529750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:18.542426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837aa1a-f217-472c-917f-d7bdda89cfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:18.543477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837aa1a-f217-472c-917f-d7bdda89cfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:18.544096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837aa1a-f217-472c-917f-d7bdda89cfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:18.544597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837aa1a-f217-472c-917f-d7bdda89cfc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:18.546165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837aa1a-f217-472c-917f-d7bdda89cfc9 None admin] RBAC: Adding query filter params (group.id=a0ade514dc644430b7241726dbed93e1, role.id=b9c95d84eaca4444b2c7b8b5eed61a59, scope.project.id=5dfc85401ad649f0811b6b4ed33a5005) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:18.546975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837aa1a-f217-472c-917f-d7bdda89cfc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:18.550321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837aa1a-f217-472c-917f-d7bdda89cfc9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:18.568418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 210/437] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Sep 23 01:06:18 2022] GET /identity/v3/role_assignments?group.id=a0ade514dc644430b7241726dbed93e1&scope.project.id=5dfc85401ad649f0811b6b4ed33a5005&role.id=b9c95d84eaca4444b2c7b8b5eed61a59 => generated 611 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:18.658144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8e26-2d98-428d-856a-f3009fc338f2 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:18.706145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10da5a-2fcc-49aa-af6f-7b6ba00d15ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:18.706867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10da5a-2fcc-49aa-af6f-7b6ba00d15ce None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:18.707150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10da5a-2fcc-49aa-af6f-7b6ba00d15ce None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:18.707429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10da5a-2fcc-49aa-af6f-7b6ba00d15ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:18.786607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e0d72-7687-4076-bee1-60883f97bf23 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:18.787024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e0d72-7687-4076-bee1-60883f97bf23 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:18.787527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e0d72-7687-4076-bee1-60883f97bf23 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:18.790358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 211/438] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:18.801670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885d08f-82a5-44ee-a25b-8d94eeca46e9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:18.802577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885d08f-82a5-44ee-a25b-8d94eeca46e9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:18.803276 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885d08f-82a5-44ee-a25b-8d94eeca46e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:18.818089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10da5a-2fcc-49aa-af6f-7b6ba00d15ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:18.819513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10da5a-2fcc-49aa-af6f-7b6ba00d15ce None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:18.834630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d885d08f-82a5-44ee-a25b-8d94eeca46e9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:18.837353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d885d08f-82a5-44ee-a25b-8d94eeca46e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:18.841086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:18.942707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 212/439] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:18.944618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885d08f-82a5-44ee-a25b-8d94eeca46e9 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:19.007585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 228/440] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:19.014160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10da5a-2fcc-49aa-af6f-7b6ba00d15ce None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:19.043632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:19.105610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885d08f-82a5-44ee-a25b-8d94eeca46e9 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:19.150701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10da5a-2fcc-49aa-af6f-7b6ba00d15ce None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:19.159875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12258eff-5ed0-4ec1-b1bf-4b16cf664160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:19.161541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12258eff-5ed0-4ec1-b1bf-4b16cf664160 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:19.162033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12258eff-5ed0-4ec1-b1bf-4b16cf664160 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:19.162724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12258eff-5ed0-4ec1-b1bf-4b16cf664160 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:19.169102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:19.179127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12258eff-5ed0-4ec1-b1bf-4b16cf664160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-12258eff-5ed0-4ec1-b1bf-4b16cf664160 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:19.184248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:19.186596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:19.187694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 213/441] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:06:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:19.195126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12258eff-5ed0-4ec1-b1bf-4b16cf664160 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:19.215809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f54f3-b969-40ad-a3ac-c683b1d4430a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:19.216858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f54f3-b969-40ad-a3ac-c683b1d4430a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:19.217655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f54f3-b969-40ad-a3ac-c683b1d4430a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:19.217975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f54f3-b969-40ad-a3ac-c683b1d4430a None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:19.219308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f54f3-b969-40ad-a3ac-c683b1d4430a None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:19.221549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f54f3-b969-40ad-a3ac-c683b1d4430a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:19.223411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 229/442] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:19 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:19.229576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f54f3-b969-40ad-a3ac-c683b1d4430a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:19.340557 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12258eff-5ed0-4ec1-b1bf-4b16cf664160 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:19.360557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f54f3-b969-40ad-a3ac-c683b1d4430a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:19.368577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:19.389754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0b0f1-2ff0-4b64-8124-d1dfb8b9e968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:19.390423 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0b0f1-2ff0-4b64-8124-d1dfb8b9e968 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:19.390748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0b0f1-2ff0-4b64-8124-d1dfb8b9e968 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:19.391021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0b0f1-2ff0-4b64-8124-d1dfb8b9e968 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:19.391750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0b0f1-2ff0-4b64-8124-d1dfb8b9e968 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:19.392054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0b0f1-2ff0-4b64-8124-d1dfb8b9e968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:19.393984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0b0f1-2ff0-4b64-8124-d1dfb8b9e968 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:19.402612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e3af-b87f-4296-9b2e-89d3c5ae9e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:19.403085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 214/443] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:19.404021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0b0f1-2ff0-4b64-8124-d1dfb8b9e968 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:19.404557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e3af-b87f-4296-9b2e-89d3c5ae9e97 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:19.404997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e3af-b87f-4296-9b2e-89d3c5ae9e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:19.405419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5e3af-b87f-4296-9b2e-89d3c5ae9e97 None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:19.407014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5e3af-b87f-4296-9b2e-89d3c5ae9e97 None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:19.409761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5e3af-b87f-4296-9b2e-89d3c5ae9e97 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:19.411969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 230/444] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:19 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:19.413386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e3af-b87f-4296-9b2e-89d3c5ae9e97 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:19.473311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:19.541920 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0b0f1-2ff0-4b64-8124-d1dfb8b9e968 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:19.550893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5e3af-b87f-4296-9b2e-89d3c5ae9e97 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:19.591757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddbc12-2661-4178-a6b7-55892b406201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:19.592350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddbc12-2661-4178-a6b7-55892b406201 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:19.592655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddbc12-2661-4178-a6b7-55892b406201 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:19.592954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddbc12-2661-4178-a6b7-55892b406201 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:19.601920 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea3cad-078f-4f58-81ae-d53281b3e62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:19.602936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea3cad-078f-4f58-81ae-d53281b3e62d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:19.603386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea3cad-078f-4f58-81ae-d53281b3e62d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:19.603887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea3cad-078f-4f58-81ae-d53281b3e62d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:19.605880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddbc12-2661-4178-a6b7-55892b406201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-10ddbc12-2661-4178-a6b7-55892b406201 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:19.612046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:19.613120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 215/445] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:06:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:19.614083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddbc12-2661-4178-a6b7-55892b406201 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:19.676484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:19.709504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea3cad-078f-4f58-81ae-d53281b3e62d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:19.711194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea3cad-078f-4f58-81ae-d53281b3e62d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:19.760055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddbc12-2661-4178-a6b7-55892b406201 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:19.773774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:19.826069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18192bd-9971-462c-a543-2deb90efc96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:19.827219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18192bd-9971-462c-a543-2deb90efc96f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:19.827848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18192bd-9971-462c-a543-2deb90efc96f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:19.828360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18192bd-9971-462c-a543-2deb90efc96f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:19.829795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18192bd-9971-462c-a543-2deb90efc96f None admin] RBAC: Adding query filter params (group.id=a0ade514dc644430b7241726dbed93e1, role.id=e5a65ae2d0884e419cdbc70c9223eac2, scope.project.id=ac99f487657f41258cd0118165066bb1) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:19.830140 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18192bd-9971-462c-a543-2deb90efc96f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:19.831881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18192bd-9971-462c-a543-2deb90efc96f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:19.849042 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 216/446] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Sep 23 01:06:19 2022] GET /identity/v3/role_assignments?group.id=a0ade514dc644430b7241726dbed93e1&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=e5a65ae2d0884e419cdbc70c9223eac2 => generated 611 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:19.849042 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18192bd-9971-462c-a543-2deb90efc96f None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:19.894469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 231/447] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:19.901819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea3cad-078f-4f58-81ae-d53281b3e62d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:20.050670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18192bd-9971-462c-a543-2deb90efc96f None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:20.063059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:20.101550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea3cad-078f-4f58-81ae-d53281b3e62d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:20.114027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab319ff9-af7a-436f-b969-b4b1c93dd6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:20.114775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab319ff9-af7a-436f-b969-b4b1c93dd6de None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:20.115066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab319ff9-af7a-436f-b969-b4b1c93dd6de None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:20.115348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab319ff9-af7a-436f-b969-b4b1c93dd6de None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:20.116112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab319ff9-af7a-436f-b969-b4b1c93dd6de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:20.116427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab319ff9-af7a-436f-b969-b4b1c93dd6de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:20.118140 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab319ff9-af7a-436f-b969-b4b1c93dd6de None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:20.129985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 217/448] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:20.136862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab319ff9-af7a-436f-b969-b4b1c93dd6de None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:20.166843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff9186-67db-4ff9-9e0d-500415d37c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:20.167117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff9186-67db-4ff9-9e0d-500415d37c62 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:20.167586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff9186-67db-4ff9-9e0d-500415d37c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:20.168151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff9186-67db-4ff9-9e0d-500415d37c62 None admin] PATH_INFO: `/v3/projects/ac99f487657f41258cd0118165066bb1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:20.170467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff9186-67db-4ff9-9e0d-500415d37c62 None admin] RBAC: Authorizing `identity:get_project(project_id=ac99f487657f41258cd0118165066bb1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:20.173385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff9186-67db-4ff9-9e0d-500415d37c62 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:20.176625 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 232/449] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:19 2022] GET /identity/v3/projects/ac99f487657f41258cd0118165066bb1 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:20.185674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff9186-67db-4ff9-9e0d-500415d37c62 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:20.213461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:20.305986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab319ff9-af7a-436f-b969-b4b1c93dd6de None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:20.369375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff9186-67db-4ff9-9e0d-500415d37c62 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:20.373994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d2b62-7620-4d2c-ae45-184d70c90df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:20.375022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d2b62-7620-4d2c-ae45-184d70c90df4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:20.375514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d2b62-7620-4d2c-ae45-184d70c90df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:20.376043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d2b62-7620-4d2c-ae45-184d70c90df4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:20.408121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302e4a4-7c90-4f4e-baba-47caf60b16e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:20.409807 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302e4a4-7c90-4f4e-baba-47caf60b16e5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:20.410093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302e4a4-7c90-4f4e-baba-47caf60b16e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:20.410366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302e4a4-7c90-4f4e-baba-47caf60b16e5 None admin] PATH_INFO: `/v3/groups/a0ade514dc644430b7241726dbed93e1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:20.411589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302e4a4-7c90-4f4e-baba-47caf60b16e5 None admin] RBAC: Authorizing `identity:get_group(group_id=a0ade514dc644430b7241726dbed93e1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:20.413118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302e4a4-7c90-4f4e-baba-47caf60b16e5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:20.415681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 233/450] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:20 2022] GET /identity/v3/groups/a0ade514dc644430b7241726dbed93e1 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:20.425112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302e4a4-7c90-4f4e-baba-47caf60b16e5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:20.436448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:20.488813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d2b62-7620-4d2c-ae45-184d70c90df4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:20.490309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d2b62-7620-4d2c-ae45-184d70c90df4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:20.525648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:20.563473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302e4a4-7c90-4f4e-baba-47caf60b16e5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:20.622715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8f714-8271-4664-a8a7-631cd5108afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:20.623704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8f714-8271-4664-a8a7-631cd5108afe None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:20.624209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8f714-8271-4664-a8a7-631cd5108afe None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:20.624702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8f714-8271-4664-a8a7-631cd5108afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:20.626079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8f714-8271-4664-a8a7-631cd5108afe None admin] RBAC: Adding query filter params (group.id=a0ade514dc644430b7241726dbed93e1, role.id=b9c95d84eaca4444b2c7b8b5eed61a59, scope.project.id=ac99f487657f41258cd0118165066bb1) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:20.626606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8f714-8271-4664-a8a7-631cd5108afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:20.629930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8f714-8271-4664-a8a7-631cd5108afe None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:20.640833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 234/451] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Sep 23 01:06:20 2022] GET /identity/v3/role_assignments?group.id=a0ade514dc644430b7241726dbed93e1&scope.project.id=ac99f487657f41258cd0118165066bb1&role.id=b9c95d84eaca4444b2c7b8b5eed61a59 => generated 611 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:20.643382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 218/452] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 01:06:20.649695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8f714-8271-4664-a8a7-631cd5108afe None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:20.788077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8f714-8271-4664-a8a7-631cd5108afe None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:20.845232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845c853-8845-4d88-876f-abb168b6b1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:20.846671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845c853-8845-4d88-876f-abb168b6b1bc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:20.847050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845c853-8845-4d88-876f-abb168b6b1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:20.847461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d845c853-8845-4d88-876f-abb168b6b1bc None admin] PATH_INFO: `/v3/projects/9c4e8ab21bdd4e5ebc7ac91f1638bec3` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:20.848839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845c853-8845-4d88-876f-abb168b6b1bc None admin] RBAC: Authorizing `identity:get_project(project_id=9c4e8ab21bdd4e5ebc7ac91f1638bec3)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:20.850336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d845c853-8845-4d88-876f-abb168b6b1bc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:20.852148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 235/453] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:06:20 2022] GET /identity/v3/projects/9c4e8ab21bdd4e5ebc7ac91f1638bec3 => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:20.860418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d2b62-7620-4d2c-ae45-184d70c90df4 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:20.982515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:21.040068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d2b62-7620-4d2c-ae45-184d70c90df4 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:21.099658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34adb9ff-cf6c-492b-a684-7d6248f74bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:21.100732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adb9ff-cf6c-492b-a684-7d6248f74bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:21.101189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adb9ff-cf6c-492b-a684-7d6248f74bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:21.101573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34adb9ff-cf6c-492b-a684-7d6248f74bc2 None admin] PATH_INFO: `/v3/groups/408a1bec2bfc4869909cf1b991f17bfb` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:21.103338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adb9ff-cf6c-492b-a684-7d6248f74bc2 None admin] RBAC: Authorizing `identity:get_group(group_id=408a1bec2bfc4869909cf1b991f17bfb)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:21.105078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34adb9ff-cf6c-492b-a684-7d6248f74bc2 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:21.107072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 219/454] 10.222.0.64 () {56 vars in 1180 bytes} [Fri Sep 23 01:06:20 2022] GET /identity/v3/groups/408a1bec2bfc4869909cf1b991f17bfb => generated 229 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:21.115762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845c853-8845-4d88-876f-abb168b6b1bc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:21.133230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:21.272586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d845c853-8845-4d88-876f-abb168b6b1bc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:21.311911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c57ea2-5e26-4aa5-900f-43756c8f2af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:21.312543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c57ea2-5e26-4aa5-900f-43756c8f2af6 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:21.312819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c57ea2-5e26-4aa5-900f-43756c8f2af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:21.313093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c57ea2-5e26-4aa5-900f-43756c8f2af6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:21.313853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c57ea2-5e26-4aa5-900f-43756c8f2af6 None admin] RBAC: Adding query filter params (group.id=408a1bec2bfc4869909cf1b991f17bfb, role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=9c4e8ab21bdd4e5ebc7ac91f1638bec3) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:21.314171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c57ea2-5e26-4aa5-900f-43756c8f2af6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:21.315712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c57ea2-5e26-4aa5-900f-43756c8f2af6 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:21.323629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 236/455] 10.222.0.64 () {56 vars in 1376 bytes} [Fri Sep 23 01:06:21 2022] GET /identity/v3/role_assignments?group.id=408a1bec2bfc4869909cf1b991f17bfb&scope.project.id=9c4e8ab21bdd4e5ebc7ac91f1638bec3&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:25.800843 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd378d86-b836-4117-bed2-671a7a1e408c None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:25.801999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd378d86-b836-4117-bed2-671a7a1e408c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:25.802597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd378d86-b836-4117-bed2-671a7a1e408c None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:25.805106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 220/456] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:25.813732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c867d6-e0c4-475e-805b-2484bee5eb0b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:25.815816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c867d6-e0c4-475e-805b-2484bee5eb0b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:25.816599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c867d6-e0c4-475e-805b-2484bee5eb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:25.865625 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-e5c867d6-e0c4-475e-805b-2484bee5eb0b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:25.867766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e5c867d6-e0c4-475e-805b-2484bee5eb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:25.906298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb510091-095a-40af-a019-856811478eca None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:25.906967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb510091-095a-40af-a019-856811478eca None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:25.907500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb510091-095a-40af-a019-856811478eca None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:25.909630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 221/457] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:25.917094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320518b5-8522-479d-8441-3b03f7aa13e3 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:25.917586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320518b5-8522-479d-8441-3b03f7aa13e3 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:25.918073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320518b5-8522-479d-8441-3b03f7aa13e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:25.963200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-320518b5-8522-479d-8441-3b03f7aa13e3 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:25.967186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-320518b5-8522-479d-8441-3b03f7aa13e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:26.007724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 237/458] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:26.009368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338cf67-e66e-4a95-8c97-4ec922939f8d None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:26.009660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338cf67-e66e-4a95-8c97-4ec922939f8d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:26.009936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338cf67-e66e-4a95-8c97-4ec922939f8d None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:26.011357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 238/459] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:26.029773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed863d-b9a1-4cbe-b2f9-256088893a35 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:26.030067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed863d-b9a1-4cbe-b2f9-256088893a35 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:26.030329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed863d-b9a1-4cbe-b2f9-256088893a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:26.046067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:26.072910 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-91ed863d-b9a1-4cbe-b2f9-256088893a35 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:26.074379 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-91ed863d-b9a1-4cbe-b2f9-256088893a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:26.097942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 222/460] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:26.154289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8eee8-f17e-467b-b45f-ea915872409b None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:26.154892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8eee8-f17e-467b-b45f-ea915872409b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:26.155374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8eee8-f17e-467b-b45f-ea915872409b None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:26.157511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 223/461] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:26.165597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae50f7-5d33-49be-8721-688cf17bd459 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:26.166339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae50f7-5d33-49be-8721-688cf17bd459 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:26.166852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ae50f7-5d33-49be-8721-688cf17bd459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:26.193030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-23ae50f7-5d33-49be-8721-688cf17bd459 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:26.194998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-23ae50f7-5d33-49be-8721-688cf17bd459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:26.250524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 239/462] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:06:26.322804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:26.338660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 224/463] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:06:26.389473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed863d-b9a1-4cbe-b2f9-256088893a35 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:26.549567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed863d-b9a1-4cbe-b2f9-256088893a35 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:26.575676 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae50f7-5d33-49be-8721-688cf17bd459 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:26.594991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438883b-3959-4d8b-9b90-601a64dc37ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:26.596223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438883b-3959-4d8b-9b90-601a64dc37ad None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:26.596705 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438883b-3959-4d8b-9b90-601a64dc37ad None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:26.597144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438883b-3959-4d8b-9b90-601a64dc37ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:26.608014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:26.610474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438883b-3959-4d8b-9b90-601a64dc37ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:26.615582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438883b-3959-4d8b-9b90-601a64dc37ad None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-c438883b-3959-4d8b-9b90-601a64dc37ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:26.633013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:26.634712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:26.636425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:26.636425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:26.636425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:26.636425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 240/464] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:26.647244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438883b-3959-4d8b-9b90-601a64dc37ad None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:26.786568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438883b-3959-4d8b-9b90-601a64dc37ad None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:26.855342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da718714-bdca-48f2-9538-2b6081383736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:26.857506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da718714-bdca-48f2-9538-2b6081383736 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:26.858243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da718714-bdca-48f2-9538-2b6081383736 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:26.858946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da718714-bdca-48f2-9538-2b6081383736 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:26.860580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da718714-bdca-48f2-9538-2b6081383736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:26.862351 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da718714-bdca-48f2-9538-2b6081383736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:26.865656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da718714-bdca-48f2-9538-2b6081383736 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:26.896115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ae50f7-5d33-49be-8721-688cf17bd459 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:26.899200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 241/465] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:26.900392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da718714-bdca-48f2-9538-2b6081383736 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:26.957542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abf8e8-b471-4a2e-a9d2-a965ed1e8a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:26.958748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abf8e8-b471-4a2e-a9d2-a965ed1e8a09 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:26.959052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abf8e8-b471-4a2e-a9d2-a965ed1e8a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:26.959419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abf8e8-b471-4a2e-a9d2-a965ed1e8a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:26.982399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abf8e8-b471-4a2e-a9d2-a965ed1e8a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:26.984094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91abf8e8-b471-4a2e-a9d2-a965ed1e8a09 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:26.996750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-91abf8e8-b471-4a2e-a9d2-a965ed1e8a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:27.003593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:27.004958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:27.006238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:27.006238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:27.006238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:27.006238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 225/466] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 430 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:27.006238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abf8e8-b471-4a2e-a9d2-a965ed1e8a09 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:27.076744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:27.091514 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da718714-bdca-48f2-9538-2b6081383736 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:27.144535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ff59-6dc3-4832-815f-af5c1d99419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:27.145218 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ff59-6dc3-4832-815f-af5c1d99419b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:27.145506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ff59-6dc3-4832-815f-af5c1d99419b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:27.145781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11ff59-6dc3-4832-815f-af5c1d99419b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:27.160166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11ff59-6dc3-4832-815f-af5c1d99419b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:27.162543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11ff59-6dc3-4832-815f-af5c1d99419b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:27.166638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91abf8e8-b471-4a2e-a9d2-a965ed1e8a09 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-bc11ff59-6dc3-4832-815f-af5c1d99419b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:27.188598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:27.191023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:27.193092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:27.193092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:27.193092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:27.193092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 242/467] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:27.193092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ff59-6dc3-4832-815f-af5c1d99419b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:27.230331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee53b6-caaa-4aeb-a6fc-5703023ef8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:27.231406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee53b6-caaa-4aeb-a6fc-5703023ef8db None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:27.232175 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee53b6-caaa-4aeb-a6fc-5703023ef8db None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:27.232175 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee53b6-caaa-4aeb-a6fc-5703023ef8db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:27.250052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee53b6-caaa-4aeb-a6fc-5703023ef8db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:27.252619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee53b6-caaa-4aeb-a6fc-5703023ef8db None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-6bee53b6-caaa-4aeb-a6fc-5703023ef8db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:27.278149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:27.280638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:27.284451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:27.284451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:27.284451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:27.284451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 226/468] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:27.284451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee53b6-caaa-4aeb-a6fc-5703023ef8db None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:27.353738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:27.362299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11ff59-6dc3-4832-815f-af5c1d99419b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:27.422318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31462c1-2b16-4cbb-9e17-7cf630692889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:27.422864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31462c1-2b16-4cbb-9e17-7cf630692889 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:27.423151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31462c1-2b16-4cbb-9e17-7cf630692889 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:27.423544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31462c1-2b16-4cbb-9e17-7cf630692889 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:27.424284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31462c1-2b16-4cbb-9e17-7cf630692889 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:27.425819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31462c1-2b16-4cbb-9e17-7cf630692889 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:27.440846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:27.444675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee53b6-caaa-4aeb-a6fc-5703023ef8db None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:27.506874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d8c2c-0c54-49b9-bbb3-2b3a2aba09a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:27.507932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d8c2c-0c54-49b9-bbb3-2b3a2aba09a7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:27.508396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d8c2c-0c54-49b9-bbb3-2b3a2aba09a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:27.508864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d8c2c-0c54-49b9-bbb3-2b3a2aba09a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:27.509778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d8c2c-0c54-49b9-bbb3-2b3a2aba09a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:27.510269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d8c2c-0c54-49b9-bbb3-2b3a2aba09a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:27.512075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d8c2c-0c54-49b9-bbb3-2b3a2aba09a7 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:27.529212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 227/469] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:27.538584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d8c2c-0c54-49b9-bbb3-2b3a2aba09a7 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:27.625188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 243/470] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:06:27 2022] POST /identity/v3/users => generated 248 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:06:27.627971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef68b97-a33f-4b84-a88a-91b47c6c5b6b None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:27.628467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef68b97-a33f-4b84-a88a-91b47c6c5b6b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:27.629022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef68b97-a33f-4b84-a88a-91b47c6c5b6b None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:27.631843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 244/471] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:27.641460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef68b97-a33f-4b84-a88a-91b47c6c5b6b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:27.729782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d8c2c-0c54-49b9-bbb3-2b3a2aba09a7 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:27.799705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eb9e7-2f93-4a18-9a4d-7b95dcdcabc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:27.799705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eb9e7-2f93-4a18-9a4d-7b95dcdcabc1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:27.800257 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eb9e7-2f93-4a18-9a4d-7b95dcdcabc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:27.800610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823eb9e7-2f93-4a18-9a4d-7b95dcdcabc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:27.801862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eb9e7-2f93-4a18-9a4d-7b95dcdcabc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:27.802419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eb9e7-2f93-4a18-9a4d-7b95dcdcabc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:27.805311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823eb9e7-2f93-4a18-9a4d-7b95dcdcabc1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:27.833120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 228/472] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:27.834602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eb9e7-2f93-4a18-9a4d-7b95dcdcabc1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:27.867470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:27.872465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef68b97-a33f-4b84-a88a-91b47c6c5b6b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:27.932355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91956d-7dfb-47f7-93e6-1f90a2e71111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:27.933564 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91956d-7dfb-47f7-93e6-1f90a2e71111 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:27.934467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91956d-7dfb-47f7-93e6-1f90a2e71111 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:27.934957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91956d-7dfb-47f7-93e6-1f90a2e71111 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:27.936197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91956d-7dfb-47f7-93e6-1f90a2e71111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:27.936633 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91956d-7dfb-47f7-93e6-1f90a2e71111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:27.938912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91956d-7dfb-47f7-93e6-1f90a2e71111 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:27.959360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 245/473] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:27.960837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669618d-3e70-4df2-87e5-14446829c11b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:27.961226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669618d-3e70-4df2-87e5-14446829c11b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:27.961509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669618d-3e70-4df2-87e5-14446829c11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:27.989581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823eb9e7-2f93-4a18-9a4d-7b95dcdcabc1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:27.995636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9669618d-3e70-4df2-87e5-14446829c11b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:27.997750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9669618d-3e70-4df2-87e5-14446829c11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:28.016990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:28.041177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93497a-d977-4e33-9aec-170807905e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:28.041863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93497a-d977-4e33-9aec-170807905e4a None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:28.042148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93497a-d977-4e33-9aec-170807905e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:28.042430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93497a-d977-4e33-9aec-170807905e4a None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:28.043108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93497a-d977-4e33-9aec-170807905e4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:28.044671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da93497a-d977-4e33-9aec-170807905e4a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:28.113856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 246/474] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:28.114801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669618d-3e70-4df2-87e5-14446829c11b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:28.142040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 229/475] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:06:27 2022] POST /identity/v3/users => generated 246 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:06:28.143890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93497a-d977-4e33-9aec-170807905e4a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:28.376132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669618d-3e70-4df2-87e5-14446829c11b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:28.383781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:28.433880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93497a-d977-4e33-9aec-170807905e4a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:28.446006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dcb32-503f-481b-a907-23df203b20cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:28.447075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dcb32-503f-481b-a907-23df203b20cb None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:28.447627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dcb32-503f-481b-a907-23df203b20cb None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:28.448119 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dcb32-503f-481b-a907-23df203b20cb None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:28.449122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dcb32-503f-481b-a907-23df203b20cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:28.450842 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dcb32-503f-481b-a907-23df203b20cb None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:28.499776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b47535-29c4-4c3c-b24f-02e8c96f0807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:28.500701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b47535-29c4-4c3c-b24f-02e8c96f0807 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:28.501162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b47535-29c4-4c3c-b24f-02e8c96f0807 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:28.501612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b47535-29c4-4c3c-b24f-02e8c96f0807 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:28.502664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b47535-29c4-4c3c-b24f-02e8c96f0807 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:28.504284 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b47535-29c4-4c3c-b24f-02e8c96f0807 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:28.543634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:28.550175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 247/476] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:06:28 2022] POST /identity/v3/users => generated 249 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:06:28.551047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dcb32-503f-481b-a907-23df203b20cb None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:28.643034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 230/477] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:06:28 2022] POST /identity/v3/users => generated 248 bytes in 500 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:06:28.760432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dcb32-503f-481b-a907-23df203b20cb None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:28.840548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9866136-8e7a-44e6-bcba-d01dab61c6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:28.842208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9866136-8e7a-44e6-bcba-d01dab61c6cf None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:28.843062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9866136-8e7a-44e6-bcba-d01dab61c6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:28.843973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9866136-8e7a-44e6-bcba-d01dab61c6cf None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:28.846159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9866136-8e7a-44e6-bcba-d01dab61c6cf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:28.849311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9866136-8e7a-44e6-bcba-d01dab61c6cf None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:28.890513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.sql.core [None req-b9866136-8e7a-44e6-bcba-d01dab61c6cf None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 23 01:06:28.890513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 23 01:06:28.890513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [parameters: {'id': '4fbb917f3fb442888f6c27994b6175ac', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 23 01:06:28.890513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81960) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-b9866136-8e7a-44e6-bcba-d01dab61c6cf None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application conn.query(q) Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application result.read() Sep 23 01:06:28.926617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return ref.to_dict() Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 01:06:28.928237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application next(self.gen) Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application yield resource Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application next(self.gen) Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self.session.rollback() Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self.force_reraise() Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise self.value Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application session.commit() Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 23 01:06:28.929965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._prepare_impl() Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self.session.flush() Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._flush(objects) Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application compat.raise_( Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application flush_context.execute() Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rec.execute(self) Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 23 01:06:28.932783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 01:06:28.936420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application conn.query(q) Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application result.read() Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 01:06:28.941369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [parameters: {'id': '4fbb917f3fb442888f6c27994b6175ac', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:28.946841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 23 01:06:28.949839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:28.952771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 248/478] 10.222.0.64 () {60 vars in 1131 bytes} [Fri Sep 23 01:06:28 2022] POST /identity/v3/roles => generated 144 bytes in 393 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:28.954665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b47535-29c4-4c3c-b24f-02e8c96f0807 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:29.144636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b47535-29c4-4c3c-b24f-02e8c96f0807 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:29.182725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:29.188106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc0109-398d-436d-a228-58b3ddced818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:29.188984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc0109-398d-436d-a228-58b3ddced818 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:29.189977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc0109-398d-436d-a228-58b3ddced818 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:29.190422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc0109-398d-436d-a228-58b3ddced818 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:29.204385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc0109-398d-436d-a228-58b3ddced818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-1abc0109-398d-436d-a228-58b3ddced818 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:29.211909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:29.213088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 23 01:06:29.214629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:29.219582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 231/479] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:06:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:29.225558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9866136-8e7a-44e6-bcba-d01dab61c6cf None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:29.340607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:29.415402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9866136-8e7a-44e6-bcba-d01dab61c6cf None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:29.480253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d0ba7-4c07-45db-a7d1-b2c3653f8190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:29.481027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d0ba7-4c07-45db-a7d1-b2c3653f8190 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:29.481389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d0ba7-4c07-45db-a7d1-b2c3653f8190 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:29.481801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d0ba7-4c07-45db-a7d1-b2c3653f8190 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:29.482773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d0ba7-4c07-45db-a7d1-b2c3653f8190 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:29.483169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d0ba7-4c07-45db-a7d1-b2c3653f8190 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:29.484997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d0ba7-4c07-45db-a7d1-b2c3653f8190 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:29.497461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 249/480] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:30.832011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449fb17-187f-43fd-a881-2580c665c498 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:30.832561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449fb17-187f-43fd-a881-2580c665c498 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:30.832652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449fb17-187f-43fd-a881-2580c665c498 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:30.834411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 232/481] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:06:30.841717 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df67bb-e10c-431d-b013-3cb6b6287dba None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:30.842381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df67bb-e10c-431d-b013-3cb6b6287dba None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:30.842903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df67bb-e10c-431d-b013-3cb6b6287dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:30.882983 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c4df67bb-e10c-431d-b013-3cb6b6287dba None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:30.885154 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c4df67bb-e10c-431d-b013-3cb6b6287dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:31.022206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 250/482] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:31.485873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449fb17-187f-43fd-a881-2580c665c498 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:31.660650 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449fb17-187f-43fd-a881-2580c665c498 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:31.702809 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0e45d1-992d-4e98-a686-4c3b5f3199d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:31.703582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e45d1-992d-4e98-a686-4c3b5f3199d7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:31.705374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e45d1-992d-4e98-a686-4c3b5f3199d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:31.705374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0e45d1-992d-4e98-a686-4c3b5f3199d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:31.717795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0e45d1-992d-4e98-a686-4c3b5f3199d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-cf0e45d1-992d-4e98-a686-4c3b5f3199d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:31.724330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:31.725349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:31.726608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 233/483] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:31.732069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df67bb-e10c-431d-b013-3cb6b6287dba None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:31.772004 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58584ea-7d65-4dab-87ec-b8d1aa551e80 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:31.772230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58584ea-7d65-4dab-87ec-b8d1aa551e80 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:31.772537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58584ea-7d65-4dab-87ec-b8d1aa551e80 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:31.774262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 234/484] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:06:31.781321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c02e9-3d0b-41c9-ad4c-1264c8820de9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:31.781598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c02e9-3d0b-41c9-ad4c-1264c8820de9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:31.781895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c02e9-3d0b-41c9-ad4c-1264c8820de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:31.869675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a51c02e9-3d0b-41c9-ad4c-1264c8820de9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:31.871576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a51c02e9-3d0b-41c9-ad4c-1264c8820de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:31.876408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:31.977535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:32.015749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 235/485] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:32.017971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661663c-d512-49b5-841e-d5e668a0a5b4 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:32.018355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661663c-d512-49b5-841e-d5e668a0a5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:32.018725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661663c-d512-49b5-841e-d5e668a0a5b4 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:32.023038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 236/486] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:32.032835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df67bb-e10c-431d-b013-3cb6b6287dba None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:32.044601 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7eba3-ad76-4652-9740-59d97e1fbf77 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:32.045050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7eba3-ad76-4652-9740-59d97e1fbf77 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:32.045523 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7eba3-ad76-4652-9740-59d97e1fbf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:32.089707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-35c7eba3-ad76-4652-9740-59d97e1fbf77 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:32.091470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-35c7eba3-ad76-4652-9740-59d97e1fbf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:32.099477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96835e59-4847-4272-9b2f-ee3d432ea0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:32.100269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96835e59-4847-4272-9b2f-ee3d432ea0ba None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:32.100556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96835e59-4847-4272-9b2f-ee3d432ea0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:32.100856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96835e59-4847-4272-9b2f-ee3d432ea0ba None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:32.101668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96835e59-4847-4272-9b2f-ee3d432ea0ba None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:32.101977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96835e59-4847-4272-9b2f-ee3d432ea0ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:32.114863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96835e59-4847-4272-9b2f-ee3d432ea0ba None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:32.135812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 251/487] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:32.143093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96835e59-4847-4272-9b2f-ee3d432ea0ba None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:32.212121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 237/488] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:06:32.375597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96835e59-4847-4272-9b2f-ee3d432ea0ba None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:32.409670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:32.438104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7eba3-ad76-4652-9740-59d97e1fbf77 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:32.454295 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c3f32-e477-43ac-9d76-84f6818341de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:32.455585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c3f32-e477-43ac-9d76-84f6818341de None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:32.455867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c3f32-e477-43ac-9d76-84f6818341de None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:32.456143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c3f32-e477-43ac-9d76-84f6818341de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:32.479238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c3f32-e477-43ac-9d76-84f6818341de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:32.481731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c3f32-e477-43ac-9d76-84f6818341de None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-6c4c3f32-e477-43ac-9d76-84f6818341de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:32.524348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:32.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:32.526874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:32.526874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:32.526874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:32.526874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 252/489] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:06:32.536719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c3f32-e477-43ac-9d76-84f6818341de None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:32.598213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7eba3-ad76-4652-9740-59d97e1fbf77 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:32.619967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:32.641995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9698395b-d774-49ad-876f-6f86a5191dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:32.641995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698395b-d774-49ad-876f-6f86a5191dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:32.642861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698395b-d774-49ad-876f-6f86a5191dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:32.643350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9698395b-d774-49ad-876f-6f86a5191dd3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:32.657340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9698395b-d774-49ad-876f-6f86a5191dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-9698395b-d774-49ad-876f-6f86a5191dd3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:32.666775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:32.667889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:32.668700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:32.672588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 238/490] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:32.679874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f2434-cd0b-4e32-a970-5f4b8c93b66a None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:32.680525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f2434-cd0b-4e32-a970-5f4b8c93b66a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:32.681784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f2434-cd0b-4e32-a970-5f4b8c93b66a None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:32.683941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 239/491] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:32.684968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f2434-cd0b-4e32-a970-5f4b8c93b66a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:32.780169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c3f32-e477-43ac-9d76-84f6818341de None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:32.841130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9fca85-b49d-4aab-a6db-93df38df1e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:32.842349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fca85-b49d-4aab-a6db-93df38df1e87 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:32.842998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fca85-b49d-4aab-a6db-93df38df1e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:32.843492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9fca85-b49d-4aab-a6db-93df38df1e87 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:32.844732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fca85-b49d-4aab-a6db-93df38df1e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:32.845874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fca85-b49d-4aab-a6db-93df38df1e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:32.848665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9fca85-b49d-4aab-a6db-93df38df1e87 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:32.853690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f2434-cd0b-4e32-a970-5f4b8c93b66a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:32.869868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 253/492] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:32.871736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bca6c-339e-40a8-a846-180235a89d0e None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:32.872259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bca6c-339e-40a8-a846-180235a89d0e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:32.872758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bca6c-339e-40a8-a846-180235a89d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:32.897040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-fd2bca6c-339e-40a8-a846-180235a89d0e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:32.900323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-fd2bca6c-339e-40a8-a846-180235a89d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:32.918805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cd81f-36b2-46a5-a019-1d9e5fb08508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:32.920530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cd81f-36b2-46a5-a019-1d9e5fb08508 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:32.920933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cd81f-36b2-46a5-a019-1d9e5fb08508 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:32.921441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cd81f-36b2-46a5-a019-1d9e5fb08508 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:32.922653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cd81f-36b2-46a5-a019-1d9e5fb08508 None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:32.923182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cd81f-36b2-46a5-a019-1d9e5fb08508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:32.926131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853cd81f-36b2-46a5-a019-1d9e5fb08508 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:32.938478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:32.942214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 240/493] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:32.943996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cd81f-36b2-46a5-a019-1d9e5fb08508 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:32.990635 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:33.043598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 254/494] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:33.043598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bca6c-339e-40a8-a846-180235a89d0e None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:33.097207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cd81f-36b2-46a5-a019-1d9e5fb08508 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:33.178311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f93abf-e60c-4e08-bad4-c89d0f19d1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:33.179741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f93abf-e60c-4e08-bad4-c89d0f19d1ae None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:33.180336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f93abf-e60c-4e08-bad4-c89d0f19d1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:33.180915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f93abf-e60c-4e08-bad4-c89d0f19d1ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:33.206726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f93abf-e60c-4e08-bad4-c89d0f19d1ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-10f93abf-e60c-4e08-bad4-c89d0f19d1ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:33.222434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:33.224530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:33.226422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 241/495] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:33.227882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f93abf-e60c-4e08-bad4-c89d0f19d1ae None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:33.238364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bca6c-339e-40a8-a846-180235a89d0e None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:33.289874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f780a-0760-4dd9-9227-f37cf8acfa98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:33.290528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f780a-0760-4dd9-9227-f37cf8acfa98 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:33.290807 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f780a-0760-4dd9-9227-f37cf8acfa98 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:33.291088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f780a-0760-4dd9-9227-f37cf8acfa98 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:33.301832 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f780a-0760-4dd9-9227-f37cf8acfa98 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:33.303316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f780a-0760-4dd9-9227-f37cf8acfa98 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-093f780a-0760-4dd9-9227-f37cf8acfa98 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:33.319150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:33.321217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 255/496] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:33.323329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f780a-0760-4dd9-9227-f37cf8acfa98 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:33.351086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:33.445492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:33.446536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f780a-0760-4dd9-9227-f37cf8acfa98 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:33.458496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f93abf-e60c-4e08-bad4-c89d0f19d1ae None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:33.501971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0f74b-e028-404f-9dc2-356424c63fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:33.503235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0f74b-e028-404f-9dc2-356424c63fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:33.503922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0f74b-e028-404f-9dc2-356424c63fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:33.504542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0f74b-e028-404f-9dc2-356424c63fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:33.505931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0f74b-e028-404f-9dc2-356424c63fe7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:33.506610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0f74b-e028-404f-9dc2-356424c63fe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:33.509411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0f74b-e028-404f-9dc2-356424c63fe7 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:33.515536 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc08ad2-73f8-4dee-9cc0-084e5ac9a4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:33.516229 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc08ad2-73f8-4dee-9cc0-084e5ac9a4ab None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:33.516504 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc08ad2-73f8-4dee-9cc0-084e5ac9a4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:33.516777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc08ad2-73f8-4dee-9cc0-084e5ac9a4ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:33.526872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 256/497] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:33.529449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0231a7-bbbe-4b76-845f-abfa8232a617 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:33.530090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0231a7-bbbe-4b76-845f-abfa8232a617 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:33.530701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0231a7-bbbe-4b76-845f-abfa8232a617 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:33.535590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 257/498] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:33.535590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0231a7-bbbe-4b76-845f-abfa8232a617 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:33.536877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc08ad2-73f8-4dee-9cc0-084e5ac9a4ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:33.538941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc08ad2-73f8-4dee-9cc0-084e5ac9a4ab None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-ebc08ad2-73f8-4dee-9cc0-084e5ac9a4ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:33.557916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:33.559768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:33.561109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:33.561109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:33.561109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:33.561109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 242/499] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:33.561109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc08ad2-73f8-4dee-9cc0-084e5ac9a4ab None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:33.646488 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:33.671455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0231a7-bbbe-4b76-845f-abfa8232a617 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:33.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc08ad2-73f8-4dee-9cc0-084e5ac9a4ab None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:33.717917 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9d20d-686f-4346-b7b1-fddd6ec17f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:33.718844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9d20d-686f-4346-b7b1-fddd6ec17f72 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:33.719197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9d20d-686f-4346-b7b1-fddd6ec17f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:33.719536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9d20d-686f-4346-b7b1-fddd6ec17f72 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:33.720437 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9d20d-686f-4346-b7b1-fddd6ec17f72 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:33.720797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9d20d-686f-4346-b7b1-fddd6ec17f72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:33.722380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9d20d-686f-4346-b7b1-fddd6ec17f72 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:33.759427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29ebc0-3549-4f72-b622-66bc202c755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:33.760119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29ebc0-3549-4f72-b622-66bc202c755d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:33.760403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29ebc0-3549-4f72-b622-66bc202c755d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:33.760712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29ebc0-3549-4f72-b622-66bc202c755d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:33.768269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 258/500] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Sep 23 01:06:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:33.770411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ef0df-426d-4df0-a5bf-f3f9da578ff5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:33.770901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ef0df-426d-4df0-a5bf-f3f9da578ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:33.771377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ef0df-426d-4df0-a5bf-f3f9da578ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:33.781524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29ebc0-3549-4f72-b622-66bc202c755d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:33.783853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29ebc0-3549-4f72-b622-66bc202c755d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:33.792030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:33.795456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-968ef0df-426d-4df0-a5bf-f3f9da578ff5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:33.796765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-968ef0df-426d-4df0-a5bf-f3f9da578ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-af29ebc0-3549-4f72-b622-66bc202c755d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:33.801277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:33.803044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:33.804722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:33.804722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:33.804722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:33.804722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 243/501] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:33.804722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29ebc0-3549-4f72-b622-66bc202c755d None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:33.903029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 259/502] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:33.904590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ef0df-426d-4df0-a5bf-f3f9da578ff5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:33.973555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29ebc0-3549-4f72-b622-66bc202c755d None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:34.021179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:34.023003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a58a5-fbf8-45fa-9e7f-244d3bc6f00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:34.023822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a58a5-fbf8-45fa-9e7f-244d3bc6f00f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:34.024161 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a58a5-fbf8-45fa-9e7f-244d3bc6f00f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:34.024410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a58a5-fbf8-45fa-9e7f-244d3bc6f00f None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:34.025096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a58a5-fbf8-45fa-9e7f-244d3bc6f00f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:34.025401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a58a5-fbf8-45fa-9e7f-244d3bc6f00f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:34.026839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a58a5-fbf8-45fa-9e7f-244d3bc6f00f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:34.051419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 244/503] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:34.051419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a58a5-fbf8-45fa-9e7f-244d3bc6f00f None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:34.096029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ef0df-426d-4df0-a5bf-f3f9da578ff5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:34.158804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:34.161915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b6a8b-83b6-48ff-8348-1c4bd41c4899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:34.163168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b6a8b-83b6-48ff-8348-1c4bd41c4899 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:34.163708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b6a8b-83b6-48ff-8348-1c4bd41c4899 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:34.164190 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b6a8b-83b6-48ff-8348-1c4bd41c4899 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:34.184399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b6a8b-83b6-48ff-8348-1c4bd41c4899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-647b6a8b-83b6-48ff-8348-1c4bd41c4899 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:34.194748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:34.195898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:34.197191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 260/504] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:34.198681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b6a8b-83b6-48ff-8348-1c4bd41c4899 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:34.242968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a58a5-fbf8-45fa-9e7f-244d3bc6f00f None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:34.300658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b76547-5371-4df8-a7c6-554e4a5db87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:34.301544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b76547-5371-4df8-a7c6-554e4a5db87c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:34.301984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b76547-5371-4df8-a7c6-554e4a5db87c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:34.302452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b76547-5371-4df8-a7c6-554e4a5db87c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:34.354510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b6a8b-83b6-48ff-8348-1c4bd41c4899 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:34.398241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:34.404179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b76547-5371-4df8-a7c6-554e4a5db87c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:34.406533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b76547-5371-4df8-a7c6-554e4a5db87c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:34.406790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03f82b-5b62-4432-9678-4848343dbf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:34.407856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03f82b-5b62-4432-9678-4848343dbf25 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:34.408305 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03f82b-5b62-4432-9678-4848343dbf25 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:34.408740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03f82b-5b62-4432-9678-4848343dbf25 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:34.409811 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03f82b-5b62-4432-9678-4848343dbf25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:34.410282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03f82b-5b62-4432-9678-4848343dbf25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:34.412309 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03f82b-5b62-4432-9678-4848343dbf25 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:34.430181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 261/505] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:34.431753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03f82b-5b62-4432-9678-4848343dbf25 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:34.498961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:34.558350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 245/506] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:34.559663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b76547-5371-4df8-a7c6-554e4a5db87c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:34.575139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03f82b-5b62-4432-9678-4848343dbf25 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:34.619792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459d6de-0618-48f5-8170-4c66b1aa4220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:34.620663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459d6de-0618-48f5-8170-4c66b1aa4220 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:34.621033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459d6de-0618-48f5-8170-4c66b1aa4220 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:34.621565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459d6de-0618-48f5-8170-4c66b1aa4220 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:34.633062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459d6de-0618-48f5-8170-4c66b1aa4220 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:34.635462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459d6de-0618-48f5-8170-4c66b1aa4220 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-a459d6de-0618-48f5-8170-4c66b1aa4220 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:34.656745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:34.658942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:34.660582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 262/507] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Sep 23 01:06:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:34.661773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459d6de-0618-48f5-8170-4c66b1aa4220 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:34.731300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:34.747980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b76547-5371-4df8-a7c6-554e4a5db87c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:34.801370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd79bb-b44b-4d29-aa67-d6c0c8563559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:34.802725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd79bb-b44b-4d29-aa67-d6c0c8563559 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:34.803224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd79bb-b44b-4d29-aa67-d6c0c8563559 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:34.803760 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd79bb-b44b-4d29-aa67-d6c0c8563559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:34.820859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459d6de-0618-48f5-8170-4c66b1aa4220 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:34.823887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd79bb-b44b-4d29-aa67-d6c0c8563559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:34.826866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd79bb-b44b-4d29-aa67-d6c0c8563559 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-4ecd79bb-b44b-4d29-aa67-d6c0c8563559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:34.859174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:34.860502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:34.861933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:34.861933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:34.861933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:34.863051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 246/508] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:34.869168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd79bb-b44b-4d29-aa67-d6c0c8563559 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:34.881574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265051c9-b4fb-40cb-a70a-66b8b1406300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:34.882328 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265051c9-b4fb-40cb-a70a-66b8b1406300 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:34.882611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265051c9-b4fb-40cb-a70a-66b8b1406300 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:34.882891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265051c9-b4fb-40cb-a70a-66b8b1406300 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:34.883713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265051c9-b4fb-40cb-a70a-66b8b1406300 None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:34.884025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265051c9-b4fb-40cb-a70a-66b8b1406300 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:34.885611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265051c9-b4fb-40cb-a70a-66b8b1406300 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:34.899224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:34.900902 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 263/509] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:34.902699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265051c9-b4fb-40cb-a70a-66b8b1406300 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:35.059645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd79bb-b44b-4d29-aa67-d6c0c8563559 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:35.096481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265051c9-b4fb-40cb-a70a-66b8b1406300 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:35.119907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:35.135071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d80dbf-4de5-4349-a88f-3ead6ebe97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:35.136643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d80dbf-4de5-4349-a88f-3ead6ebe97de None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:35.136911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d80dbf-4de5-4349-a88f-3ead6ebe97de None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:35.137401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d80dbf-4de5-4349-a88f-3ead6ebe97de None admin] PATH_INFO: `/v3/users/glance` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:35.152664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d80dbf-4de5-4349-a88f-3ead6ebe97de None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:35.155587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d80dbf-4de5-4349-a88f-3ead6ebe97de None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:35.174390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694121a0-b651-4dfb-9bcd-6313f4f1fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:35.175258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694121a0-b651-4dfb-9bcd-6313f4f1fac6 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:35.175588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694121a0-b651-4dfb-9bcd-6313f4f1fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:35.175872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694121a0-b651-4dfb-9bcd-6313f4f1fac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-d0d80dbf-4de5-4349-a88f-3ead6ebe97de None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:35.180907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:35.182703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:35.184438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 247/510] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:35.185534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d80dbf-4de5-4349-a88f-3ead6ebe97de None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:35.194812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694121a0-b651-4dfb-9bcd-6313f4f1fac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:35.197732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694121a0-b651-4dfb-9bcd-6313f4f1fac6 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-694121a0-b651-4dfb-9bcd-6313f4f1fac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:35.242004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:35.243597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:35.244448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:35.244448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:35.244448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:35.244448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 264/511] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:35.244786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694121a0-b651-4dfb-9bcd-6313f4f1fac6 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:35.317186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:35.356073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d80dbf-4de5-4349-a88f-3ead6ebe97de None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:35.389428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694121a0-b651-4dfb-9bcd-6313f4f1fac6 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:35.411103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9aaf9-706e-4d4d-8d3b-0ac70ccc4e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:35.411773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9aaf9-706e-4d4d-8d3b-0ac70ccc4e19 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:35.412059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9aaf9-706e-4d4d-8d3b-0ac70ccc4e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:35.412334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c9aaf9-706e-4d4d-8d3b-0ac70ccc4e19 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:35.413123 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9aaf9-706e-4d4d-8d3b-0ac70ccc4e19 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:35.413523 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9aaf9-706e-4d4d-8d3b-0ac70ccc4e19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:35.415176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c9aaf9-706e-4d4d-8d3b-0ac70ccc4e19 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:35.433301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28d793-4cf1-4a35-bb8e-37031d3b723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:35.434012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28d793-4cf1-4a35-bb8e-37031d3b723d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:35.434349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28d793-4cf1-4a35-bb8e-37031d3b723d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:35.434626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28d793-4cf1-4a35-bb8e-37031d3b723d None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:35.435298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28d793-4cf1-4a35-bb8e-37031d3b723d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:35.435740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28d793-4cf1-4a35-bb8e-37031d3b723d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:35.437419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28d793-4cf1-4a35-bb8e-37031d3b723d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:35.456516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 265/512] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:35.457968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28d793-4cf1-4a35-bb8e-37031d3b723d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:35.492026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 248/513] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Sep 23 01:06:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:35.492026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9aaf9-706e-4d4d-8d3b-0ac70ccc4e19 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:35.492302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:35.585238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28d793-4cf1-4a35-bb8e-37031d3b723d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:35.631626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0582a-286f-43e0-a7c8-1f9c9b76785c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:35.632481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0582a-286f-43e0-a7c8-1f9c9b76785c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:35.632849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0582a-286f-43e0-a7c8-1f9c9b76785c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:35.633211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0582a-286f-43e0-a7c8-1f9c9b76785c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:35.646112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0582a-286f-43e0-a7c8-1f9c9b76785c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:35.647987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0582a-286f-43e0-a7c8-1f9c9b76785c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:35.658479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c9aaf9-706e-4d4d-8d3b-0ac70ccc4e19 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-1be0582a-286f-43e0-a7c8-1f9c9b76785c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:35.668319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:35.670366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:35.672289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:35.672289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:35.672289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:35.672781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 266/514] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:35.674877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0582a-286f-43e0-a7c8-1f9c9b76785c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:35.696787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:35.708960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c73590-9a24-4560-a01e-6defd9d90869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:35.710007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c73590-9a24-4560-a01e-6defd9d90869 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:35.710521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c73590-9a24-4560-a01e-6defd9d90869 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:35.711892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c73590-9a24-4560-a01e-6defd9d90869 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:35.712401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c73590-9a24-4560-a01e-6defd9d90869 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:35.712929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c73590-9a24-4560-a01e-6defd9d90869 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:35.715584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c73590-9a24-4560-a01e-6defd9d90869 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:35.748591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 249/515] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Sep 23 01:06:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:35.749553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c73590-9a24-4560-a01e-6defd9d90869 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:35.780267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:35.816523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0582a-286f-43e0-a7c8-1f9c9b76785c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:35.862729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98428295-0625-43a7-8ba7-2d54cc17df29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:35.863264 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98428295-0625-43a7-8ba7-2d54cc17df29 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:35.863770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98428295-0625-43a7-8ba7-2d54cc17df29 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:35.864204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98428295-0625-43a7-8ba7-2d54cc17df29 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:35.864898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98428295-0625-43a7-8ba7-2d54cc17df29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:35.865311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98428295-0625-43a7-8ba7-2d54cc17df29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:35.866748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98428295-0625-43a7-8ba7-2d54cc17df29 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:35.881094 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 267/516] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:35.881981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98428295-0625-43a7-8ba7-2d54cc17df29 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:35.912541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c73590-9a24-4560-a01e-6defd9d90869 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:35.954022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561d23e-134c-4439-8cdd-68111ef7fa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:35.955157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561d23e-134c-4439-8cdd-68111ef7fa18 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:35.955452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561d23e-134c-4439-8cdd-68111ef7fa18 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:35.955856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561d23e-134c-4439-8cdd-68111ef7fa18 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:35.956541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561d23e-134c-4439-8cdd-68111ef7fa18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:35.958038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561d23e-134c-4439-8cdd-68111ef7fa18 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:36.019046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98428295-0625-43a7-8ba7-2d54cc17df29 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:36.058071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 250/517] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:06:35 2022] POST /identity/v3/users => generated 247 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:06:36.059564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561d23e-134c-4439-8cdd-68111ef7fa18 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:36.067283 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d31b5b-44a9-4212-b52e-980d28a9f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:36.068045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d31b5b-44a9-4212-b52e-980d28a9f420 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:36.068452 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d31b5b-44a9-4212-b52e-980d28a9f420 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:36.069003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d31b5b-44a9-4212-b52e-980d28a9f420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:36.073940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:36.136296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:36.195387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d31b5b-44a9-4212-b52e-980d28a9f420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:36.197883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d31b5b-44a9-4212-b52e-980d28a9f420 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:36.231967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561d23e-134c-4439-8cdd-68111ef7fa18 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:36.288292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa0c9b-052b-4963-b1f5-2a086dcae6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:36.289358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0c9b-052b-4963-b1f5-2a086dcae6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:36.289851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0c9b-052b-4963-b1f5-2a086dcae6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:36.290335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0c9b-052b-4963-b1f5-2a086dcae6c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:36.291581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa0c9b-052b-4963-b1f5-2a086dcae6c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:36.293082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa0c9b-052b-4963-b1f5-2a086dcae6c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:36.295859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa0c9b-052b-4963-b1f5-2a086dcae6c0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:36.317796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 251/518] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:36.319041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa0c9b-052b-4963-b1f5-2a086dcae6c0 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:36.397161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 268/519] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:36.403586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d31b5b-44a9-4212-b52e-980d28a9f420 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:36.453232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:36.513959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa0c9b-052b-4963-b1f5-2a086dcae6c0 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:36.515876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:36.554268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d31b5b-44a9-4212-b52e-980d28a9f420 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:36.570737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24629b43-cf2e-4b79-96d6-1c4f398e6e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:36.571365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24629b43-cf2e-4b79-96d6-1c4f398e6e06 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:36.571692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24629b43-cf2e-4b79-96d6-1c4f398e6e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:36.571962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24629b43-cf2e-4b79-96d6-1c4f398e6e06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:36.615428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929feddb-6117-4eeb-90a1-20f42acda64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:36.616553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929feddb-6117-4eeb-90a1-20f42acda64f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:36.617022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929feddb-6117-4eeb-90a1-20f42acda64f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:36.617483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929feddb-6117-4eeb-90a1-20f42acda64f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:36.629983 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929feddb-6117-4eeb-90a1-20f42acda64f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:36.632712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929feddb-6117-4eeb-90a1-20f42acda64f None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-929feddb-6117-4eeb-90a1-20f42acda64f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:36.654629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:36.656368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:36.664197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:36.664197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:36.664197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:36.664197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:36.664197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:36.664197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:36.664197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:36.664197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:36.664197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 269/520] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:36.664197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929feddb-6117-4eeb-90a1-20f42acda64f None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:36.669511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24629b43-cf2e-4b79-96d6-1c4f398e6e06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:36.671999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24629b43-cf2e-4b79-96d6-1c4f398e6e06 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:36.822745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 252/521] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:36.828683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24629b43-cf2e-4b79-96d6-1c4f398e6e06 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:36.831513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929feddb-6117-4eeb-90a1-20f42acda64f None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:36.878274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c70320-a62f-417d-bf58-f0c0175603aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:36.878946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c70320-a62f-417d-bf58-f0c0175603aa None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:36.879234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c70320-a62f-417d-bf58-f0c0175603aa None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:36.879516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c70320-a62f-417d-bf58-f0c0175603aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:36.891790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c70320-a62f-417d-bf58-f0c0175603aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:36.893641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c70320-a62f-417d-bf58-f0c0175603aa None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-d4c70320-a62f-417d-bf58-f0c0175603aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:36.907692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:36.908992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:36.910398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:36.910398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:36.910398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:36.910398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:36.910398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:36.910398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:36.910398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:36.910398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:36.910398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 01:06:36.910398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:36.914252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 270/522] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:36.922975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c70320-a62f-417d-bf58-f0c0175603aa None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:36.922975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:36.975266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24629b43-cf2e-4b79-96d6-1c4f398e6e06 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:37.041435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f65541-4899-4273-90f4-c9748af88fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:37.042767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f65541-4899-4273-90f4-c9748af88fea None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:37.043394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f65541-4899-4273-90f4-c9748af88fea None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:37.044055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f65541-4899-4273-90f4-c9748af88fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:37.063825 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c70320-a62f-417d-bf58-f0c0175603aa None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:37.064571 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f65541-4899-4273-90f4-c9748af88fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:37.066661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f65541-4899-4273-90f4-c9748af88fea None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-88f65541-4899-4273-90f4-c9748af88fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:37.095662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:37.097030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:37.100431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:37.100431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:37.100431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:37.100431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 253/523] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:37.100431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f65541-4899-4273-90f4-c9748af88fea None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:37.117604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5e448-74c5-4e1c-9bea-68f9d496956e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:37.146020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5e448-74c5-4e1c-9bea-68f9d496956e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:37.146020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5e448-74c5-4e1c-9bea-68f9d496956e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:37.146020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5e448-74c5-4e1c-9bea-68f9d496956e None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:37.146020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5e448-74c5-4e1c-9bea-68f9d496956e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:37.146020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5e448-74c5-4e1c-9bea-68f9d496956e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:37.146020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5e448-74c5-4e1c-9bea-68f9d496956e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:37.155619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 271/524] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:37.155619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5e448-74c5-4e1c-9bea-68f9d496956e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:37.237723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:37.299388 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f65541-4899-4273-90f4-c9748af88fea None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:37.324843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:37.345211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5e448-74c5-4e1c-9bea-68f9d496956e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:37.359417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608dd09-fdbb-4a94-96b7-556d7b536592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:37.360108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608dd09-fdbb-4a94-96b7-556d7b536592 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:37.360390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608dd09-fdbb-4a94-96b7-556d7b536592 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:37.360692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3608dd09-fdbb-4a94-96b7-556d7b536592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:37.378865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608dd09-fdbb-4a94-96b7-556d7b536592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:37.381427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3608dd09-fdbb-4a94-96b7-556d7b536592 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:37.396755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:37.409693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7804b-e98b-40b1-a6cb-9d9f50bbcb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:37.410766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7804b-e98b-40b1-a6cb-9d9f50bbcb8d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:37.411245 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7804b-e98b-40b1-a6cb-9d9f50bbcb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:37.411742 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7804b-e98b-40b1-a6cb-9d9f50bbcb8d None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-3608dd09-fdbb-4a94-96b7-556d7b536592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:37.412126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:37.413146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:37.420139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:37.420139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:37.420139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:37.420404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7804b-e98b-40b1-a6cb-9d9f50bbcb8d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:37.420404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7804b-e98b-40b1-a6cb-9d9f50bbcb8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:37.420534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 254/525] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:37.420534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608dd09-fdbb-4a94-96b7-556d7b536592 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:37.420697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca7804b-e98b-40b1-a6cb-9d9f50bbcb8d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:37.452851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 272/526] 10.222.0.64 () {56 vars in 1139 bytes} [Fri Sep 23 01:06:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:37.455257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7804b-e98b-40b1-a6cb-9d9f50bbcb8d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:37.574216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:37.591330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3608dd09-fdbb-4a94-96b7-556d7b536592 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:37.599133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:37.608823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7804b-e98b-40b1-a6cb-9d9f50bbcb8d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:37.631168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa2466-ee5a-4098-991e-b0dc4da8964b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:37.631928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa2466-ee5a-4098-991e-b0dc4da8964b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:37.632212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa2466-ee5a-4098-991e-b0dc4da8964b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:37.632494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa2466-ee5a-4098-991e-b0dc4da8964b None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:37.633178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa2466-ee5a-4098-991e-b0dc4da8964b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:37.633488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa2466-ee5a-4098-991e-b0dc4da8964b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:37.634902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fa2466-ee5a-4098-991e-b0dc4da8964b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:37.653516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 255/527] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:37.653516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa2466-ee5a-4098-991e-b0dc4da8964b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:37.658698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448344c8-c55c-45a1-a077-7cb8d0d1600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:37.659528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448344c8-c55c-45a1-a077-7cb8d0d1600e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:37.659928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448344c8-c55c-45a1-a077-7cb8d0d1600e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:37.660270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448344c8-c55c-45a1-a077-7cb8d0d1600e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:37.661139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448344c8-c55c-45a1-a077-7cb8d0d1600e None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=51ae9d865bb4442490ca61b18d1f7ec7) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:37.661522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448344c8-c55c-45a1-a077-7cb8d0d1600e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:37.663096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448344c8-c55c-45a1-a077-7cb8d0d1600e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:37.670666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 273/528] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:06:37 2022] GET /identity/v3/role_assignments?user.id=51ae9d865bb4442490ca61b18d1f7ec7&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:37.681037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448344c8-c55c-45a1-a077-7cb8d0d1600e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:37.805597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa2466-ee5a-4098-991e-b0dc4da8964b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:37.856670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdd468-85f4-441f-addb-7dd0d14eca30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:37.857906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdd468-85f4-441f-addb-7dd0d14eca30 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:37.858386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdd468-85f4-441f-addb-7dd0d14eca30 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:37.858842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fdd468-85f4-441f-addb-7dd0d14eca30 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:37.860475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdd468-85f4-441f-addb-7dd0d14eca30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:37.861614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdd468-85f4-441f-addb-7dd0d14eca30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:37.863782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fdd468-85f4-441f-addb-7dd0d14eca30 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:37.874257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448344c8-c55c-45a1-a077-7cb8d0d1600e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:37.882326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 256/529] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:37.883629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdd468-85f4-441f-addb-7dd0d14eca30 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:37.922514 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cea5e-79b9-4c50-9459-4334ed1c5a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:37.924018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cea5e-79b9-4c50-9459-4334ed1c5a72 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:37.924375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cea5e-79b9-4c50-9459-4334ed1c5a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:37.924831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cea5e-79b9-4c50-9459-4334ed1c5a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:37.929970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:37.938779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:38.034670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cea5e-79b9-4c50-9459-4334ed1c5a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:38.037089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cea5e-79b9-4c50-9459-4334ed1c5a72 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:38.067475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fdd468-85f4-441f-addb-7dd0d14eca30 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:38.128605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09192a-5075-430d-97f2-ea87b6af4908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:38.129270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09192a-5075-430d-97f2-ea87b6af4908 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:38.129716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09192a-5075-430d-97f2-ea87b6af4908 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:38.130202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09192a-5075-430d-97f2-ea87b6af4908 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:38.141120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09192a-5075-430d-97f2-ea87b6af4908 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:38.144122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09192a-5075-430d-97f2-ea87b6af4908 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-1a09192a-5075-430d-97f2-ea87b6af4908 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:38.163089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:38.164169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:38.165037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:38.165037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:38.165037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:38.165037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:38.165037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:38.165037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:38.165037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:38.165037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:38.174050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 257/530] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:38.174050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09192a-5075-430d-97f2-ea87b6af4908 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:38.201699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 274/531] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:38.203313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cea5e-79b9-4c50-9459-4334ed1c5a72 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:38.284119 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:38.355310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:38.366811 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cea5e-79b9-4c50-9459-4334ed1c5a72 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:38.390258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09192a-5075-430d-97f2-ea87b6af4908 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:38.424179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ae043-e28a-44b5-b619-6c91f4f72d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:38.424936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ae043-e28a-44b5-b619-6c91f4f72d22 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:38.425317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ae043-e28a-44b5-b619-6c91f4f72d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:38.425603 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ae043-e28a-44b5-b619-6c91f4f72d22 None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:38.426423 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ae043-e28a-44b5-b619-6c91f4f72d22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:38.426744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ae043-e28a-44b5-b619-6c91f4f72d22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:38.428750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ae043-e28a-44b5-b619-6c91f4f72d22 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:38.450971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed4c68-31ad-4b9d-aa2c-65da83893501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:38.458172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 275/532] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:38.458172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ae043-e28a-44b5-b619-6c91f4f72d22 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:38.458473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed4c68-31ad-4b9d-aa2c-65da83893501 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:38.458473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed4c68-31ad-4b9d-aa2c-65da83893501 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:38.458473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed4c68-31ad-4b9d-aa2c-65da83893501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:38.475394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed4c68-31ad-4b9d-aa2c-65da83893501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:38.479609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed4c68-31ad-4b9d-aa2c-65da83893501 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-1aed4c68-31ad-4b9d-aa2c-65da83893501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:38.507214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:38.508376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:38.509408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:38.509408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:38.509408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:38.509408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:38.509408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:38.509408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:38.509408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:38.509408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:38.510696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 258/533] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:38.512612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed4c68-31ad-4b9d-aa2c-65da83893501 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:38.669902 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:38.679388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ae043-e28a-44b5-b619-6c91f4f72d22 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:38.686000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed4c68-31ad-4b9d-aa2c-65da83893501 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:38.739721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d4000-ef40-42f1-b5c1-093577b85e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:38.741133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d4000-ef40-42f1-b5c1-093577b85e58 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:38.741626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d4000-ef40-42f1-b5c1-093577b85e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:38.742114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d4000-ef40-42f1-b5c1-093577b85e58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:38.745219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc3088-36ae-4ae8-a489-d4067d5df1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:38.750705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc3088-36ae-4ae8-a489-d4067d5df1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:38.750705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc3088-36ae-4ae8-a489-d4067d5df1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:38.750705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc3088-36ae-4ae8-a489-d4067d5df1c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:38.750705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc3088-36ae-4ae8-a489-d4067d5df1c1 None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=1c4da5b2130d4fa990adb72f0f6ec2fe) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:38.750705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc3088-36ae-4ae8-a489-d4067d5df1c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:38.750705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc3088-36ae-4ae8-a489-d4067d5df1c1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:38.760858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d4000-ef40-42f1-b5c1-093577b85e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:38.761980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 259/534] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:06:38 2022] GET /identity/v3/role_assignments?user.id=1c4da5b2130d4fa990adb72f0f6ec2fe&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:38.763766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc3088-36ae-4ae8-a489-d4067d5df1c1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:38.764089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d4000-ef40-42f1-b5c1-093577b85e58 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-2f2d4000-ef40-42f1-b5c1-093577b85e58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:38.792029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:38.793070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:38.794066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:38.794066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:38.794066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:38.796598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 276/535] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:38.799242 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:38.805078 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d4000-ef40-42f1-b5c1-093577b85e58 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:39.037994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc3088-36ae-4ae8-a489-d4067d5df1c1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:39.057607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d4000-ef40-42f1-b5c1-093577b85e58 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:39.093051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc672b8-700f-4b19-90dd-5e9dd6191d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:39.094003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc672b8-700f-4b19-90dd-5e9dd6191d31 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:39.094706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc672b8-700f-4b19-90dd-5e9dd6191d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:39.095156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc672b8-700f-4b19-90dd-5e9dd6191d31 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:39.096533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc672b8-700f-4b19-90dd-5e9dd6191d31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:39.096938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc672b8-700f-4b19-90dd-5e9dd6191d31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:39.099589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc672b8-700f-4b19-90dd-5e9dd6191d31 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:39.111245 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f657f4-4aed-44a1-a4e8-8c765e98bb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:39.112467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f657f4-4aed-44a1-a4e8-8c765e98bb66 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:39.115081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f657f4-4aed-44a1-a4e8-8c765e98bb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:39.115081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f657f4-4aed-44a1-a4e8-8c765e98bb66 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:39.115081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f657f4-4aed-44a1-a4e8-8c765e98bb66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:39.115363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f657f4-4aed-44a1-a4e8-8c765e98bb66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:39.117980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f657f4-4aed-44a1-a4e8-8c765e98bb66 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:39.124143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 260/536] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:39.125520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb06b0-c35d-4057-9c53-a118bdc8883c None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:39.125781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb06b0-c35d-4057-9c53-a118bdc8883c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:39.126066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb06b0-c35d-4057-9c53-a118bdc8883c None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:39.128155 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 261/537] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:39.135399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8b6ec-5ad9-4286-abcc-578c08ed382c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:39.135814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8b6ec-5ad9-4286-abcc-578c08ed382c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:39.136716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8b6ec-5ad9-4286-abcc-578c08ed382c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:39.142634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 277/538] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:39.150868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f657f4-4aed-44a1-a4e8-8c765e98bb66 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:39.159733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-bbe8b6ec-5ad9-4286-abcc-578c08ed382c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:39.161092 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-bbe8b6ec-5ad9-4286-abcc-578c08ed382c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:39.172206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:39.299771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:39.304998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f657f4-4aed-44a1-a4e8-8c765e98bb66 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:39.310139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 262/539] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:06:39.311471 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8b6ec-5ad9-4286-abcc-578c08ed382c None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:39.404581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393433e-fdf9-4d89-ab82-0c3fb6bc0c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:39.405122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393433e-fdf9-4d89-ab82-0c3fb6bc0c8f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:39.405384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393433e-fdf9-4d89-ab82-0c3fb6bc0c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:39.405649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393433e-fdf9-4d89-ab82-0c3fb6bc0c8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:39.406497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393433e-fdf9-4d89-ab82-0c3fb6bc0c8f None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=6f10c258fe5f45ccab1669fa32ed6029) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:39.406817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393433e-fdf9-4d89-ab82-0c3fb6bc0c8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:39.408878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393433e-fdf9-4d89-ab82-0c3fb6bc0c8f None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:39.422782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 278/540] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:06:39 2022] GET /identity/v3/role_assignments?user.id=6f10c258fe5f45ccab1669fa32ed6029&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:39.505526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8b6ec-5ad9-4286-abcc-578c08ed382c None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:39.572096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d8e04-1e89-4bdc-bbfd-85ec4afba2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:39.573490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8e04-1e89-4bdc-bbfd-85ec4afba2bd None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:39.574010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8e04-1e89-4bdc-bbfd-85ec4afba2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:39.574517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d8e04-1e89-4bdc-bbfd-85ec4afba2bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:39.593523 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d8e04-1e89-4bdc-bbfd-85ec4afba2bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:39.595588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d8e04-1e89-4bdc-bbfd-85ec4afba2bd None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-242d8e04-1e89-4bdc-bbfd-85ec4afba2bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:39.609975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:39.611566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:39.612617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:39.612617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:39.612617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:39.612617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:39.612617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:39.612617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:39.612617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:39.612617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:39.616903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 263/541] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:39.625814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393433e-fdf9-4d89-ab82-0c3fb6bc0c8f None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:39.708513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d8e04-1e89-4bdc-bbfd-85ec4afba2bd None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:39.804466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:39.836211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393433e-fdf9-4d89-ab82-0c3fb6bc0c8f None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:39.871719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:39.889824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e5350-8874-4afc-a549-814161cbe4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:39.890905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e5350-8874-4afc-a549-814161cbe4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:39.891435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e5350-8874-4afc-a549-814161cbe4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:39.892595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e5350-8874-4afc-a549-814161cbe4e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:39.893818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e5350-8874-4afc-a549-814161cbe4e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:39.894417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e5350-8874-4afc-a549-814161cbe4e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:39.896222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e5350-8874-4afc-a549-814161cbe4e6 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:39.912551 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 279/542] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:39.922012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e5350-8874-4afc-a549-814161cbe4e6 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:39.996958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d8e04-1e89-4bdc-bbfd-85ec4afba2bd None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:40.107005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc039d-bb21-4406-97a1-f16bd4494fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:40.107005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc039d-bb21-4406-97a1-f16bd4494fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:40.107005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc039d-bb21-4406-97a1-f16bd4494fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:40.107005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc039d-bb21-4406-97a1-f16bd4494fa5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:40.123589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e5350-8874-4afc-a549-814161cbe4e6 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:40.123849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc039d-bb21-4406-97a1-f16bd4494fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-debc039d-bb21-4406-97a1-f16bd4494fa5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:40.137733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:40.139386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:40.145673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 264/543] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:06:40.168727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc039d-bb21-4406-97a1-f16bd4494fa5 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:40.201885 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03bf9f-cee2-4da1-bcc2-ff7ad370f9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:40.203182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03bf9f-cee2-4da1-bcc2-ff7ad370f9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:40.204011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03bf9f-cee2-4da1-bcc2-ff7ad370f9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:40.205338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03bf9f-cee2-4da1-bcc2-ff7ad370f9f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:40.208437 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03bf9f-cee2-4da1-bcc2-ff7ad370f9f8 None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=3b0a7b4f607d4753bf142e1420bcd987) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:40.209073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03bf9f-cee2-4da1-bcc2-ff7ad370f9f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:40.211332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03bf9f-cee2-4da1-bcc2-ff7ad370f9f8 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:40.217331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:40.228827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 280/544] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:06:39 2022] GET /identity/v3/role_assignments?user.id=3b0a7b4f607d4753bf142e1420bcd987&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:40.409549 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc039d-bb21-4406-97a1-f16bd4494fa5 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:40.446242 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:40.477926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66f3ab-3af0-4ad2-97a4-91b84235c5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:40.478672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66f3ab-3af0-4ad2-97a4-91b84235c5be None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:40.479060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66f3ab-3af0-4ad2-97a4-91b84235c5be None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:40.479359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66f3ab-3af0-4ad2-97a4-91b84235c5be None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:40.480123 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66f3ab-3af0-4ad2-97a4-91b84235c5be None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:40.480430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66f3ab-3af0-4ad2-97a4-91b84235c5be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:40.481988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66f3ab-3af0-4ad2-97a4-91b84235c5be None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:40.500739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 265/545] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:40.508090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03bf9f-cee2-4da1-bcc2-ff7ad370f9f8 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:40.735963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03bf9f-cee2-4da1-bcc2-ff7ad370f9f8 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:40.811596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3757aac-8b54-4e67-ae41-ec7434b6028f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:40.812683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3757aac-8b54-4e67-ae41-ec7434b6028f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:40.813033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3757aac-8b54-4e67-ae41-ec7434b6028f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:40.813393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3757aac-8b54-4e67-ae41-ec7434b6028f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:40.839096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3757aac-8b54-4e67-ae41-ec7434b6028f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:40.842086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3757aac-8b54-4e67-ae41-ec7434b6028f None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-f3757aac-8b54-4e67-ae41-ec7434b6028f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:40.884428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:40.885441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:40.886307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:40.886307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:40.886307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:40.886485 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 281/546] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:06:40.894564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66f3ab-3af0-4ad2-97a4-91b84235c5be None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:40.956399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a293f2-e0ca-4293-8718-35a4beaaa53c None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:40.957684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a293f2-e0ca-4293-8718-35a4beaaa53c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:40.957684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a293f2-e0ca-4293-8718-35a4beaaa53c None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:40.961579 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 282/547] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:40.977989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031eb98-dc95-4712-8eaf-fa00eb559651 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:40.978480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031eb98-dc95-4712-8eaf-fa00eb559651 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:40.978963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5031eb98-dc95-4712-8eaf-fa00eb559651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:41.032002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5031eb98-dc95-4712-8eaf-fa00eb559651 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:41.034773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5031eb98-dc95-4712-8eaf-fa00eb559651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:41.204102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:41.218480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66f3ab-3af0-4ad2-97a4-91b84235c5be None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:41.262726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 283/548] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:41.274238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d449c-c9c7-4324-aa68-1112962537ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:41.274880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d449c-c9c7-4324-aa68-1112962537ec None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:41.275169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d449c-c9c7-4324-aa68-1112962537ec None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:41.275480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d449c-c9c7-4324-aa68-1112962537ec None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:41.276192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d449c-c9c7-4324-aa68-1112962537ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:41.276510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d449c-c9c7-4324-aa68-1112962537ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:41.277926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20d449c-c9c7-4324-aa68-1112962537ec None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:41.302817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 266/549] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:41.308764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031eb98-dc95-4712-8eaf-fa00eb559651 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:41.491838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5031eb98-dc95-4712-8eaf-fa00eb559651 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:41.562259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639e18a-f5a9-48ce-bfdf-743418107846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:41.562963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639e18a-f5a9-48ce-bfdf-743418107846 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:41.563223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639e18a-f5a9-48ce-bfdf-743418107846 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:41.563504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639e18a-f5a9-48ce-bfdf-743418107846 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:41.578743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639e18a-f5a9-48ce-bfdf-743418107846 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:41.580845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639e18a-f5a9-48ce-bfdf-743418107846 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-8639e18a-f5a9-48ce-bfdf-743418107846 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:41.611004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:41.612045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:41.613014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:41.613014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:41.613014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:41.613014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:41.613014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:41.613014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:41.613014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:41.613014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:41.613014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:06:41.613014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:41.614169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 284/550] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:06:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:06:41.623205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d449c-c9c7-4324-aa68-1112962537ec None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:41.701096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:41.813417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20d449c-c9c7-4324-aa68-1112962537ec None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:41.872138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639e18a-f5a9-48ce-bfdf-743418107846 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:41.891598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb255bef-fc16-44c5-8866-0409ad18a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:41.892365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb255bef-fc16-44c5-8866-0409ad18a21a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:41.892663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb255bef-fc16-44c5-8866-0409ad18a21a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:41.892954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb255bef-fc16-44c5-8866-0409ad18a21a None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:41.894049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb255bef-fc16-44c5-8866-0409ad18a21a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:41.894509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb255bef-fc16-44c5-8866-0409ad18a21a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:41.896398 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb255bef-fc16-44c5-8866-0409ad18a21a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:41.936095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 267/551] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Sep 23 01:06:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:41.940222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb255bef-fc16-44c5-8866-0409ad18a21a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:41.952170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:42.020296 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639e18a-f5a9-48ce-bfdf-743418107846 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:42.076977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff4833-6aac-4da0-aff2-4a8001d54b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:42.077655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff4833-6aac-4da0-aff2-4a8001d54b15 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:42.078001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff4833-6aac-4da0-aff2-4a8001d54b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:42.078354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff4833-6aac-4da0-aff2-4a8001d54b15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:42.101187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff4833-6aac-4da0-aff2-4a8001d54b15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-fbff4833-6aac-4da0-aff2-4a8001d54b15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:42.106868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:42.108037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:42.108944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 285/552] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:06:42.123326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff4833-6aac-4da0-aff2-4a8001d54b15 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:42.203499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:42.214077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb255bef-fc16-44c5-8866-0409ad18a21a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:42.274029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ec6ad-ad53-49ec-8bca-7e3883ab37a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:42.274792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ec6ad-ad53-49ec-8bca-7e3883ab37a7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:42.275078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ec6ad-ad53-49ec-8bca-7e3883ab37a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:42.275347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ec6ad-ad53-49ec-8bca-7e3883ab37a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:42.340915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff4833-6aac-4da0-aff2-4a8001d54b15 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:42.358281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:42.412404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236fb81-ad32-4eca-957e-4cc4d0987822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:42.413350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236fb81-ad32-4eca-957e-4cc4d0987822 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:42.413784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236fb81-ad32-4eca-957e-4cc4d0987822 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:42.414217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236fb81-ad32-4eca-957e-4cc4d0987822 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:42.415334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236fb81-ad32-4eca-957e-4cc4d0987822 None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:42.415865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236fb81-ad32-4eca-957e-4cc4d0987822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:42.418295 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236fb81-ad32-4eca-957e-4cc4d0987822 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:42.426877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ec6ad-ad53-49ec-8bca-7e3883ab37a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:42.429358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ec6ad-ad53-49ec-8bca-7e3883ab37a7 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:42.464817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 286/553] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:42.477626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236fb81-ad32-4eca-957e-4cc4d0987822 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:42.691677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:42.696140 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 268/554] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 757 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 01:06:42.697559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf5e0b-c9eb-4079-876a-71dec305958c None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:42.697846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf5e0b-c9eb-4079-876a-71dec305958c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:42.698122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf5e0b-c9eb-4079-876a-71dec305958c None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:42.699791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 269/555] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:42.701522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf5e0b-c9eb-4079-876a-71dec305958c None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:42.721833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236fb81-ad32-4eca-957e-4cc4d0987822 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:42.789276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf88dc-cbbe-4f35-9e86-be70762e478c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:42.790491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf88dc-cbbe-4f35-9e86-be70762e478c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:42.791125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf88dc-cbbe-4f35-9e86-be70762e478c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:42.791647 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf88dc-cbbe-4f35-9e86-be70762e478c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:42.827588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf88dc-cbbe-4f35-9e86-be70762e478c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:42.827588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf88dc-cbbe-4f35-9e86-be70762e478c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:42.840311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf5e0b-c9eb-4079-876a-71dec305958c None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-3ccf88dc-cbbe-4f35-9e86-be70762e478c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:42.857478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:42.858506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:42.860164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:42.860164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:42.860164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:42.860164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 287/556] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:06:42.865251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13654866-c8d0-4bfa-a43e-ef6a2b25f048 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:42.866144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13654866-c8d0-4bfa-a43e-ef6a2b25f048 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:42.866671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13654866-c8d0-4bfa-a43e-ef6a2b25f048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:42.894693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9798bbe-5ae7-4eea-80ca-ab66d48ec603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:42.896011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9798bbe-5ae7-4eea-80ca-ab66d48ec603 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:42.896672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9798bbe-5ae7-4eea-80ca-ab66d48ec603 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:42.897655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9798bbe-5ae7-4eea-80ca-ab66d48ec603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:42.905084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-13654866-c8d0-4bfa-a43e-ef6a2b25f048 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:42.907746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-13654866-c8d0-4bfa-a43e-ef6a2b25f048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:42.916799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9798bbe-5ae7-4eea-80ca-ab66d48ec603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:42.920570 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9798bbe-5ae7-4eea-80ca-ab66d48ec603 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:42.922743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-a9798bbe-5ae7-4eea-80ca-ab66d48ec603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:42.945470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:42.947039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:42.948044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:42.948044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:42.948044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:42.950020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 270/557] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:42.951191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9798bbe-5ae7-4eea-80ca-ab66d48ec603 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:43.024204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 288/558] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:43.025614 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13654866-c8d0-4bfa-a43e-ef6a2b25f048 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:43.097048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:43.238769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9798bbe-5ae7-4eea-80ca-ab66d48ec603 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:43.243932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13654866-c8d0-4bfa-a43e-ef6a2b25f048 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:43.295469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8222f4-218b-43ec-a63a-67a4948f95f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:43.296844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8222f4-218b-43ec-a63a-67a4948f95f2 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:43.297371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8222f4-218b-43ec-a63a-67a4948f95f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:43.298522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8222f4-218b-43ec-a63a-67a4948f95f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:43.299837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8222f4-218b-43ec-a63a-67a4948f95f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:43.300406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8222f4-218b-43ec-a63a-67a4948f95f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:43.303065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8222f4-218b-43ec-a63a-67a4948f95f2 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:43.335790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b75ad-ec96-4dbe-9806-39b21fb9f8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:43.336828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b75ad-ec96-4dbe-9806-39b21fb9f8db None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:43.338269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 289/559] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:43.338840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b75ad-ec96-4dbe-9806-39b21fb9f8db None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:43.339331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b75ad-ec96-4dbe-9806-39b21fb9f8db None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:43.339510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbbca5-9f5f-4b8f-99ad-fd4a7f46d166 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:43.340042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbbca5-9f5f-4b8f-99ad-fd4a7f46d166 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:43.340533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbbca5-9f5f-4b8f-99ad-fd4a7f46d166 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:43.340933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b75ad-ec96-4dbe-9806-39b21fb9f8db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:43.341435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b75ad-ec96-4dbe-9806-39b21fb9f8db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:43.343314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b75ad-ec96-4dbe-9806-39b21fb9f8db None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:43.343491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 290/560] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:43.345761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbbca5-9f5f-4b8f-99ad-fd4a7f46d166 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:43.353036 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:43.371759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 271/561] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:43.374369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848940-b42d-4846-85ad-c02473ead8ba None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:43.375007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848940-b42d-4846-85ad-c02473ead8ba None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:43.375908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9848940-b42d-4846-85ad-c02473ead8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:43.409995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b9848940-b42d-4846-85ad-c02473ead8ba None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:43.412385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b9848940-b42d-4846-85ad-c02473ead8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:43.551001 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 272/562] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:43.552999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9848940-b42d-4846-85ad-c02473ead8ba None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:43.576872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbbca5-9f5f-4b8f-99ad-fd4a7f46d166 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:43.620045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435a686-5253-4d61-aad2-8f6425fad1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:43.620958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435a686-5253-4d61-aad2-8f6425fad1ec None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:43.621604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435a686-5253-4d61-aad2-8f6425fad1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:43.622031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435a686-5253-4d61-aad2-8f6425fad1ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:43.631893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435a686-5253-4d61-aad2-8f6425fad1ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:43.633797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435a686-5253-4d61-aad2-8f6425fad1ec None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:43.649787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-d435a686-5253-4d61-aad2-8f6425fad1ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:43.650330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:43.651207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:43.652240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:43.652240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:43.652240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:43.652240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:43.652240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:43.652240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:43.652240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:43.652240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:43.654821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 291/563] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:43.657032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435a686-5253-4d61-aad2-8f6425fad1ec None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:43.756145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:43.779256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9848940-b42d-4846-85ad-c02473ead8ba None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:43.826833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435a686-5253-4d61-aad2-8f6425fad1ec None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:43.837874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73452de9-8e72-4d00-91d6-d11a41d4a7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:43.838891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73452de9-8e72-4d00-91d6-d11a41d4a7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:43.839464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73452de9-8e72-4d00-91d6-d11a41d4a7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:43.840068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73452de9-8e72-4d00-91d6-d11a41d4a7c1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:43.851256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73452de9-8e72-4d00-91d6-d11a41d4a7c1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:43.853266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73452de9-8e72-4d00-91d6-d11a41d4a7c1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-73452de9-8e72-4d00-91d6-d11a41d4a7c1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:43.867862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:43.868911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:43.869981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:43.869981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:43.869981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:43.869981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:43.869981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:43.869981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:43.869981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:43.869981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:43.869981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 01:06:43.869981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:43.872148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 273/564] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:43.873209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73452de9-8e72-4d00-91d6-d11a41d4a7c1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:43.894163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a6299-92e1-4153-aa04-486ed7d226cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:43.895466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6299-92e1-4153-aa04-486ed7d226cc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:43.896537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6299-92e1-4153-aa04-486ed7d226cc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:43.896537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a6299-92e1-4153-aa04-486ed7d226cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:43.912962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a6299-92e1-4153-aa04-486ed7d226cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-bc9a6299-92e1-4153-aa04-486ed7d226cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:43.919981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:43.921866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 292/565] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:43.924180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc3958-b68a-4d17-ab75-01330415bb53 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:43.928368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc3958-b68a-4d17-ab75-01330415bb53 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:43.928368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc3958-b68a-4d17-ab75-01330415bb53 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:43.928368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 293/566] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:43.928368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc3958-b68a-4d17-ab75-01330415bb53 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:44.050190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73452de9-8e72-4d00-91d6-d11a41d4a7c1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:44.053344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:44.083597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc3958-b68a-4d17-ab75-01330415bb53 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:44.099107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef447-5e55-42a3-ae87-c049c69dcb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:44.100102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef447-5e55-42a3-ae87-c049c69dcb12 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:44.101272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef447-5e55-42a3-ae87-c049c69dcb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:44.101937 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddef447-5e55-42a3-ae87-c049c69dcb12 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:44.103184 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef447-5e55-42a3-ae87-c049c69dcb12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:44.103495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef447-5e55-42a3-ae87-c049c69dcb12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:44.106190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddef447-5e55-42a3-ae87-c049c69dcb12 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:44.115229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:44.119924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 274/567] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:44.120789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef447-5e55-42a3-ae87-c049c69dcb12 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:44.132001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafcd658-bda8-46da-961f-7c843536e44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:44.134061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcd658-bda8-46da-961f-7c843536e44c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:44.134061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcd658-bda8-46da-961f-7c843536e44c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:44.134061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcd658-bda8-46da-961f-7c843536e44c None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:44.134446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafcd658-bda8-46da-961f-7c843536e44c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:44.134573 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafcd658-bda8-46da-961f-7c843536e44c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:44.136038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafcd658-bda8-46da-961f-7c843536e44c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:44.156297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 294/568] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Sep 23 01:06:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:44.157671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fed4d-b9ec-4cfd-a393-88b04209fee8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:44.157955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fed4d-b9ec-4cfd-a393-88b04209fee8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:44.158237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fed4d-b9ec-4cfd-a393-88b04209fee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:44.180854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f46fed4d-b9ec-4cfd-a393-88b04209fee8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:44.182139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f46fed4d-b9ec-4cfd-a393-88b04209fee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:44.257107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddef447-5e55-42a3-ae87-c049c69dcb12 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:44.296106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 295/569] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:44.296106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fed4d-b9ec-4cfd-a393-88b04209fee8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:44.308933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19a341-477b-4677-8be2-3c361ec7966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:44.309555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19a341-477b-4677-8be2-3c361ec7966b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:44.309866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19a341-477b-4677-8be2-3c361ec7966b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:44.310262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19a341-477b-4677-8be2-3c361ec7966b None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:44.311016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19a341-477b-4677-8be2-3c361ec7966b None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:44.311330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19a341-477b-4677-8be2-3c361ec7966b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:44.312981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19a341-477b-4677-8be2-3c361ec7966b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:44.328592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 275/570] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:44.328592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19a341-477b-4677-8be2-3c361ec7966b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:44.361455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:44.459687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fed4d-b9ec-4cfd-a393-88b04209fee8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:44.480296 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:44.499897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19a341-477b-4677-8be2-3c361ec7966b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:44.517573 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b421667-2f8e-4837-933e-60a012bfbf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:44.518703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b421667-2f8e-4837-933e-60a012bfbf34 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:44.519210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b421667-2f8e-4837-933e-60a012bfbf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:44.519748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b421667-2f8e-4837-933e-60a012bfbf34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:44.540852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b421667-2f8e-4837-933e-60a012bfbf34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:44.548827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b50f4-a338-4477-9a68-c95bb04420db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:44.549857 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b50f4-a338-4477-9a68-c95bb04420db None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:44.550288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b50f4-a338-4477-9a68-c95bb04420db None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:44.550779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b50f4-a338-4477-9a68-c95bb04420db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-0b421667-2f8e-4837-933e-60a012bfbf34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:44.551034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:44.553039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:44.554635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:44.555928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b50f4-a338-4477-9a68-c95bb04420db None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=c5a9f3eca3e34fffa7b218064e831d7b) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:44.555928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b50f4-a338-4477-9a68-c95bb04420db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:44.555928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b50f4-a338-4477-9a68-c95bb04420db None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:44.561135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 296/571] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:44.564144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b421667-2f8e-4837-933e-60a012bfbf34 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:44.570182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 276/572] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:06:44 2022] GET /identity/v3/role_assignments?user.id=c5a9f3eca3e34fffa7b218064e831d7b&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:44.572082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b50f4-a338-4477-9a68-c95bb04420db None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:44.745783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:44.765744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b421667-2f8e-4837-933e-60a012bfbf34 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:44.770235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b50f4-a338-4477-9a68-c95bb04420db None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:44.841327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d3e37-aa9f-448f-9f78-ce374a8249a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:44.842205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d3e37-aa9f-448f-9f78-ce374a8249a5 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:44.842811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d3e37-aa9f-448f-9f78-ce374a8249a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:44.842970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef956cc-db29-4837-8b73-5e1ff147d6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:44.843123 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511d3e37-aa9f-448f-9f78-ce374a8249a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:44.843850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef956cc-db29-4837-8b73-5e1ff147d6bc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:44.844253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef956cc-db29-4837-8b73-5e1ff147d6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:44.844648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef956cc-db29-4837-8b73-5e1ff147d6bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:44.864219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d3e37-aa9f-448f-9f78-ce374a8249a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:44.864896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef956cc-db29-4837-8b73-5e1ff147d6bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:44.866727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511d3e37-aa9f-448f-9f78-ce374a8249a5 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:44.867424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef956cc-db29-4837-8b73-5e1ff147d6bc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-511d3e37-aa9f-448f-9f78-ce374a8249a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:44.888745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:44.890559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:44.897299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:44.897299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:44.897299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:44.897299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 277/573] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:44.897299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d3e37-aa9f-448f-9f78-ce374a8249a5 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-6ef956cc-db29-4837-8b73-5e1ff147d6bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:44.901345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:44.904907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:44.908923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:44.908923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:44.908923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:44.908923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 297/574] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:44.912765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef956cc-db29-4837-8b73-5e1ff147d6bc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:44.946452 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:45.048886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef956cc-db29-4837-8b73-5e1ff147d6bc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:45.081812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511d3e37-aa9f-448f-9f78-ce374a8249a5 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:45.106150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2503153-1b22-40fe-9d5a-98e1e0433266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:45.107850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2503153-1b22-40fe-9d5a-98e1e0433266 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:45.108302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2503153-1b22-40fe-9d5a-98e1e0433266 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:45.109104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2503153-1b22-40fe-9d5a-98e1e0433266 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:45.127199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2503153-1b22-40fe-9d5a-98e1e0433266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-c2503153-1b22-40fe-9d5a-98e1e0433266 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:45.134507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:45.135775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 298/575] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:45.137378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2503153-1b22-40fe-9d5a-98e1e0433266 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:45.139989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13da5f-caa9-471b-92b9-05ab2813362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:45.140662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13da5f-caa9-471b-92b9-05ab2813362a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:45.141050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13da5f-caa9-471b-92b9-05ab2813362a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:45.141314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13da5f-caa9-471b-92b9-05ab2813362a None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:45.142551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13da5f-caa9-471b-92b9-05ab2813362a None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:45.142863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13da5f-caa9-471b-92b9-05ab2813362a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:45.144363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13da5f-caa9-471b-92b9-05ab2813362a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:45.162032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 278/576] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:45.162032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13da5f-caa9-471b-92b9-05ab2813362a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:45.164704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:45.272934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:45.303172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2503153-1b22-40fe-9d5a-98e1e0433266 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:45.315257 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13da5f-caa9-471b-92b9-05ab2813362a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:45.352537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d7eac-eef8-43b6-a216-ecdb636fd128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:45.353170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d7eac-eef8-43b6-a216-ecdb636fd128 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:45.353608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d7eac-eef8-43b6-a216-ecdb636fd128 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:45.354124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d7eac-eef8-43b6-a216-ecdb636fd128 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:45.355336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d7eac-eef8-43b6-a216-ecdb636fd128 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:45.355537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d7eac-eef8-43b6-a216-ecdb636fd128 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:45.357098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d7eac-eef8-43b6-a216-ecdb636fd128 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:45.364576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e98b60-e31d-4f00-8eef-2f3fadf4567f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:45.365376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e98b60-e31d-4f00-8eef-2f3fadf4567f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:45.365837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e98b60-e31d-4f00-8eef-2f3fadf4567f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:45.366293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e98b60-e31d-4f00-8eef-2f3fadf4567f None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:45.367170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e98b60-e31d-4f00-8eef-2f3fadf4567f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:45.367597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e98b60-e31d-4f00-8eef-2f3fadf4567f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:45.369253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e98b60-e31d-4f00-8eef-2f3fadf4567f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:45.373648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 299/577] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:45.374873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d7eac-eef8-43b6-a216-ecdb636fd128 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:45.390489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 279/578] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:45.391402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e98b60-e31d-4f00-8eef-2f3fadf4567f None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:45.476928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:45.524898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e98b60-e31d-4f00-8eef-2f3fadf4567f None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:45.549003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d7eac-eef8-43b6-a216-ecdb636fd128 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:45.577003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a439c1-0d77-4f21-a936-b70371dd78aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:45.577421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a439c1-0d77-4f21-a936-b70371dd78aa None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:45.577667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a439c1-0d77-4f21-a936-b70371dd78aa None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:45.577935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a439c1-0d77-4f21-a936-b70371dd78aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:45.592909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a439c1-0d77-4f21-a936-b70371dd78aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:45.594413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a439c1-0d77-4f21-a936-b70371dd78aa None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-14a439c1-0d77-4f21-a936-b70371dd78aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:45.612228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:45.613455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:45.614858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:45.614858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:45.614858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:45.614858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 280/579] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:45.614858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a439c1-0d77-4f21-a936-b70371dd78aa None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:45.617966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103204b6-83e1-418f-8c94-29252725ec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:45.618674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103204b6-83e1-418f-8c94-29252725ec4c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:45.618960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103204b6-83e1-418f-8c94-29252725ec4c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:45.619248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103204b6-83e1-418f-8c94-29252725ec4c None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:45.620112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103204b6-83e1-418f-8c94-29252725ec4c None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:45.620378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103204b6-83e1-418f-8c94-29252725ec4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:45.622439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103204b6-83e1-418f-8c94-29252725ec4c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:45.633873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 300/580] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:45.634518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103204b6-83e1-418f-8c94-29252725ec4c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:45.651470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:45.776886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a439c1-0d77-4f21-a936-b70371dd78aa None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:45.794928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103204b6-83e1-418f-8c94-29252725ec4c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:45.799158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:45.839432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce6d07-fdd4-4d3f-b84a-99a71b8139a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:45.840690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce6d07-fdd4-4d3f-b84a-99a71b8139a3 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:45.841039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce6d07-fdd4-4d3f-b84a-99a71b8139a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:45.841241 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce6d07-fdd4-4d3f-b84a-99a71b8139a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:45.851236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce6d07-fdd4-4d3f-b84a-99a71b8139a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:45.852680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5714d51-89a4-4bde-857f-e34581fdd2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:45.853051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce6d07-fdd4-4d3f-b84a-99a71b8139a3 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:45.853345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714d51-89a4-4bde-857f-e34581fdd2dc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:45.853694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714d51-89a4-4bde-857f-e34581fdd2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:45.854024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5714d51-89a4-4bde-857f-e34581fdd2dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:45.862277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5714d51-89a4-4bde-857f-e34581fdd2dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:45.864074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5714d51-89a4-4bde-857f-e34581fdd2dc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-a6ce6d07-fdd4-4d3f-b84a-99a71b8139a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:45.874331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:45.876615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:45.879649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:45.879649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:45.879649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:45.879649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:45.879649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:45.879649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:45.879649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:45.879649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:45.879649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:06:45.879649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:45.881452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 281/581] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Sep 23 01:06:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:45.882041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce6d07-fdd4-4d3f-b84a-99a71b8139a3 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-f5714d51-89a4-4bde-857f-e34581fdd2dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:45.888107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:45.889819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:45.891522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:45.891522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:45.891522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:45.891522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:45.891522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:45.891522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:45.891522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:45.891522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:45.891522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 301/582] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:45.894808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5714d51-89a4-4bde-857f-e34581fdd2dc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:46.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:46.018923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce6d07-fdd4-4d3f-b84a-99a71b8139a3 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:46.054822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5714d51-89a4-4bde-857f-e34581fdd2dc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:46.066050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cca402-8da7-4e60-9275-2b53481a5090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:46.066535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cca402-8da7-4e60-9275-2b53481a5090 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:46.066904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cca402-8da7-4e60-9275-2b53481a5090 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:46.067191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cca402-8da7-4e60-9275-2b53481a5090 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:46.072393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cca402-8da7-4e60-9275-2b53481a5090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:46.072393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cca402-8da7-4e60-9275-2b53481a5090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:46.072393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cca402-8da7-4e60-9275-2b53481a5090 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:46.085517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 282/583] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:46.086464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cca402-8da7-4e60-9275-2b53481a5090 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:46.105732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8c02f-9366-4966-97d6-893a2a72693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:46.106390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8c02f-9366-4966-97d6-893a2a72693b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:46.106673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8c02f-9366-4966-97d6-893a2a72693b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:46.106947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8c02f-9366-4966-97d6-893a2a72693b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:46.124291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8c02f-9366-4966-97d6-893a2a72693b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:46.126487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8c02f-9366-4966-97d6-893a2a72693b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-bcb8c02f-9366-4966-97d6-893a2a72693b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:46.148438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:46.150085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:46.151313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:46.151313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:46.151313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:46.156180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 302/584] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:46.156372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:46.157019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8c02f-9366-4966-97d6-893a2a72693b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:46.265717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cca402-8da7-4e60-9275-2b53481a5090 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:46.308401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8c02f-9366-4966-97d6-893a2a72693b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:46.313333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106eb33-7f41-40c6-9275-7cc7a9dc31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:46.314622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106eb33-7f41-40c6-9275-7cc7a9dc31ef None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:46.314969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106eb33-7f41-40c6-9275-7cc7a9dc31ef None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:46.315325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106eb33-7f41-40c6-9275-7cc7a9dc31ef None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:46.316192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106eb33-7f41-40c6-9275-7cc7a9dc31ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:46.316569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106eb33-7f41-40c6-9275-7cc7a9dc31ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:46.318134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106eb33-7f41-40c6-9275-7cc7a9dc31ef None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:46.344499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 283/585] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Sep 23 01:06:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:46.344499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106eb33-7f41-40c6-9275-7cc7a9dc31ef None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:46.347763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:46.378758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883037a-e3cb-49e6-8654-7192f014d5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:46.379492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883037a-e3cb-49e6-8654-7192f014d5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:46.379881 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883037a-e3cb-49e6-8654-7192f014d5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:46.380232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883037a-e3cb-49e6-8654-7192f014d5d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:46.381173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883037a-e3cb-49e6-8654-7192f014d5d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:46.381703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883037a-e3cb-49e6-8654-7192f014d5d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:46.383886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883037a-e3cb-49e6-8654-7192f014d5d4 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:46.403242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 303/586] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:46.404838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883037a-e3cb-49e6-8654-7192f014d5d4 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:46.507514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:46.520240 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106eb33-7f41-40c6-9275-7cc7a9dc31ef None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:46.572904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd2d01-5808-43c0-920e-3901c9f19f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:46.573839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd2d01-5808-43c0-920e-3901c9f19f6f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:46.574277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd2d01-5808-43c0-920e-3901c9f19f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:46.574681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd2d01-5808-43c0-920e-3901c9f19f6f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:46.584391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883037a-e3cb-49e6-8654-7192f014d5d4 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:46.585197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd2d01-5808-43c0-920e-3901c9f19f6f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:46.587052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd2d01-5808-43c0-920e-3901c9f19f6f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-65fd2d01-5808-43c0-920e-3901c9f19f6f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:46.603014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:46.604868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 284/587] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:46.608702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd2d01-5808-43c0-920e-3901c9f19f6f None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:46.663752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2134eaa-1a5e-4a50-bf9e-5998d28278fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:46.665303 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2134eaa-1a5e-4a50-bf9e-5998d28278fb None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:46.666001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2134eaa-1a5e-4a50-bf9e-5998d28278fb None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:46.666474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2134eaa-1a5e-4a50-bf9e-5998d28278fb None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:46.667788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2134eaa-1a5e-4a50-bf9e-5998d28278fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:46.668624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2134eaa-1a5e-4a50-bf9e-5998d28278fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:46.671712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2134eaa-1a5e-4a50-bf9e-5998d28278fb None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:46.702728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 304/588] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:46.702728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2134eaa-1a5e-4a50-bf9e-5998d28278fb None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:46.778206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:46.801539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd2d01-5808-43c0-920e-3901c9f19f6f None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:46.863647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4915b7-78d4-45bb-a538-370b50210171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:46.864213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4915b7-78d4-45bb-a538-370b50210171 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:46.864494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4915b7-78d4-45bb-a538-370b50210171 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:46.864772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4915b7-78d4-45bb-a538-370b50210171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:46.878217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4915b7-78d4-45bb-a538-370b50210171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:46.879979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4915b7-78d4-45bb-a538-370b50210171 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:46.898516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2134eaa-1a5e-4a50-bf9e-5998d28278fb None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-ee4915b7-78d4-45bb-a538-370b50210171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:46.901637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:46.902690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:46.903866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:46.903866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:46.903866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:46.904945 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 285/589] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:46.905972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4915b7-78d4-45bb-a538-370b50210171 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:46.910803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:46.958672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4f9ff-bb0b-4881-a89c-6d98452e9025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:46.959454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4f9ff-bb0b-4881-a89c-6d98452e9025 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:46.959769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4f9ff-bb0b-4881-a89c-6d98452e9025 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:46.960046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4f9ff-bb0b-4881-a89c-6d98452e9025 None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/51ae9d865bb4442490ca61b18d1f7ec7/roles/50ceaff2140f4e05a56877fee2f609e4` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:46.983499 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4f9ff-bb0b-4881-a89c-6d98452e9025 None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=51ae9d865bb4442490ca61b18d1f7ec7, role_id=50ceaff2140f4e05a56877fee2f609e4)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:46.986508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4f9ff-bb0b-4881-a89c-6d98452e9025 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:47.050641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4915b7-78d4-45bb-a538-370b50210171 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:47.093712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c1ec2-c72d-4c4c-a8f6-8ea56da5c597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:47.095446 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c1ec2-c72d-4c4c-a8f6-8ea56da5c597 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:47.095707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c1ec2-c72d-4c4c-a8f6-8ea56da5c597 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:47.096021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c1ec2-c72d-4c4c-a8f6-8ea56da5c597 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:47.096913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c1ec2-c72d-4c4c-a8f6-8ea56da5c597 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:47.097317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c1ec2-c72d-4c4c-a8f6-8ea56da5c597 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:47.098885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c1ec2-c72d-4c4c-a8f6-8ea56da5c597 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:47.107950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 305/590] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:06:46 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/51ae9d865bb4442490ca61b18d1f7ec7/roles/50ceaff2140f4e05a56877fee2f609e4 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:06:47.107950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4f9ff-bb0b-4881-a89c-6d98452e9025 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:47.138698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 286/591] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Sep 23 01:06:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:47.138698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c1ec2-c72d-4c4c-a8f6-8ea56da5c597 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:47.241623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:47.253985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:47.273611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4f9ff-bb0b-4881-a89c-6d98452e9025 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:47.302422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c1ec2-c72d-4c4c-a8f6-8ea56da5c597 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:47.342768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58f9e1-0730-44e4-b75a-0b201beb7c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:47.344443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58f9e1-0730-44e4-b75a-0b201beb7c0e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:47.345352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58f9e1-0730-44e4-b75a-0b201beb7c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:47.345804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58f9e1-0730-44e4-b75a-0b201beb7c0e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:47.355434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d58f9e1-0730-44e4-b75a-0b201beb7c0e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:47.357988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d58f9e1-0730-44e4-b75a-0b201beb7c0e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:47.360288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a7551-a673-426a-a1f0-2632921c5bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:47.361786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a7551-a673-426a-a1f0-2632921c5bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:47.362279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a7551-a673-426a-a1f0-2632921c5bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:47.362927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98a7551-a673-426a-a1f0-2632921c5bf9 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:47.364427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a7551-a673-426a-a1f0-2632921c5bf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:47.365306 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a7551-a673-426a-a1f0-2632921c5bf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:47.368290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98a7551-a673-426a-a1f0-2632921c5bf9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-3d58f9e1-0730-44e4-b75a-0b201beb7c0e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:47.372993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:47.374738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:47.376091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 306/592] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:47.380511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58f9e1-0730-44e4-b75a-0b201beb7c0e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:47.402095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 287/593] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:47.402095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a7551-a673-426a-a1f0-2632921c5bf9 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:47.548071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58f9e1-0730-44e4-b75a-0b201beb7c0e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:47.591342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98a7551-a673-426a-a1f0-2632921c5bf9 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:47.598578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8f490-81be-4a69-b520-9847f46688c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:47.599687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8f490-81be-4a69-b520-9847f46688c5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:47.600171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8f490-81be-4a69-b520-9847f46688c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:47.600643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc8f490-81be-4a69-b520-9847f46688c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:47.615693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8f490-81be-4a69-b520-9847f46688c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:47.617375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc8f490-81be-4a69-b520-9847f46688c5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:47.624122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:47.628994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:47.638039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a532dd1-0971-43ca-b15d-54c5f7f064e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:47.638940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a532dd1-0971-43ca-b15d-54c5f7f064e6 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:47.639376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a532dd1-0971-43ca-b15d-54c5f7f064e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:47.639938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a532dd1-0971-43ca-b15d-54c5f7f064e6 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:47.647080 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a532dd1-0971-43ca-b15d-54c5f7f064e6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:47.647080 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a532dd1-0971-43ca-b15d-54c5f7f064e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-0fc8f490-81be-4a69-b520-9847f46688c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:47.647343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:47.648781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:47.649963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:47.649963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:47.649963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:47.650191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a532dd1-0971-43ca-b15d-54c5f7f064e6 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:47.650290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 307/594] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:47.650290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8f490-81be-4a69-b520-9847f46688c5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:47.675826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 288/595] 10.222.0.64 () {56 vars in 1139 bytes} [Fri Sep 23 01:06:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:47.675826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30225db-82d1-41ef-95b6-aaeb332fa216 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:47.675826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30225db-82d1-41ef-95b6-aaeb332fa216 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:47.675826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30225db-82d1-41ef-95b6-aaeb332fa216 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:47.675826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 289/596] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:47.675826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30225db-82d1-41ef-95b6-aaeb332fa216 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:47.826353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc8f490-81be-4a69-b520-9847f46688c5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:47.853814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30225db-82d1-41ef-95b6-aaeb332fa216 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:47.888370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d97f8-ea07-4b5c-9cfc-da85e9843315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:47.889384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d97f8-ea07-4b5c-9cfc-da85e9843315 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:47.889853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d97f8-ea07-4b5c-9cfc-da85e9843315 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:47.890299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d97f8-ea07-4b5c-9cfc-da85e9843315 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:47.904187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d97f8-ea07-4b5c-9cfc-da85e9843315 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:47.906681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d97f8-ea07-4b5c-9cfc-da85e9843315 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:47.914443 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6000118-1c88-4106-92e8-7cc5110d5d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:47.915706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6000118-1c88-4106-92e8-7cc5110d5d63 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:47.916308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6000118-1c88-4106-92e8-7cc5110d5d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:47.916889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6000118-1c88-4106-92e8-7cc5110d5d63 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:47.918059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6000118-1c88-4106-92e8-7cc5110d5d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:47.918672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6000118-1c88-4106-92e8-7cc5110d5d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:47.921234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6000118-1c88-4106-92e8-7cc5110d5d63 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-3f3d97f8-ea07-4b5c-9cfc-da85e9843315 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:47.927951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:47.929831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:47.931835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:47.931835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:47.931835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:47.931835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:47.931835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:47.931835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:47.931835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:47.931835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:47.931835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 308/597] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:47.931835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d97f8-ea07-4b5c-9cfc-da85e9843315 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:47.954532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 290/598] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:47.956588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff0f1-1d38-4bcb-9667-c16345182203 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:47.957063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff0f1-1d38-4bcb-9667-c16345182203 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:47.957520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dff0f1-1d38-4bcb-9667-c16345182203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:47.988034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-98dff0f1-1d38-4bcb-9667-c16345182203 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:47.990166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-98dff0f1-1d38-4bcb-9667-c16345182203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:48.017736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:48.048335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:48.077271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d97f8-ea07-4b5c-9cfc-da85e9843315 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:48.100971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 291/599] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:48.102554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dff0f1-1d38-4bcb-9667-c16345182203 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:48.154004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee9826-f03c-4aac-ba2d-11f8509aab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:48.155159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee9826-f03c-4aac-ba2d-11f8509aab66 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:48.155774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee9826-f03c-4aac-ba2d-11f8509aab66 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:48.156237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ee9826-f03c-4aac-ba2d-11f8509aab66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:48.181262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee9826-f03c-4aac-ba2d-11f8509aab66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:48.183704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ee9826-f03c-4aac-ba2d-11f8509aab66 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-67ee9826-f03c-4aac-ba2d-11f8509aab66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:48.214957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:48.216815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:48.218431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:48.218431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:48.218431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:48.228403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 309/600] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:48.230580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee9826-f03c-4aac-ba2d-11f8509aab66 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:48.275530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dff0f1-1d38-4bcb-9667-c16345182203 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:48.338517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09512bd0-0fc3-402d-9dd0-710a8b78e4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:48.339421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09512bd0-0fc3-402d-9dd0-710a8b78e4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:48.339882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09512bd0-0fc3-402d-9dd0-710a8b78e4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:48.340301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09512bd0-0fc3-402d-9dd0-710a8b78e4a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:48.341511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09512bd0-0fc3-402d-9dd0-710a8b78e4a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:48.341968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09512bd0-0fc3-402d-9dd0-710a8b78e4a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:48.344421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09512bd0-0fc3-402d-9dd0-710a8b78e4a9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:48.361298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 292/601] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:48.362488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09512bd0-0fc3-402d-9dd0-710a8b78e4a9 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:48.403117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:48.411616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ee9826-f03c-4aac-ba2d-11f8509aab66 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:48.430417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:48.467981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7122be4-4efa-4923-bf3e-568c90ff43bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:48.468878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7122be4-4efa-4923-bf3e-568c90ff43bc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:48.469344 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7122be4-4efa-4923-bf3e-568c90ff43bc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:48.469665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7122be4-4efa-4923-bf3e-568c90ff43bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:48.478611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7122be4-4efa-4923-bf3e-568c90ff43bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:48.482078 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7122be4-4efa-4923-bf3e-568c90ff43bc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-c7122be4-4efa-4923-bf3e-568c90ff43bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:48.503228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:48.504235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:48.505300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:48.505300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:48.505300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:48.505300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:48.505300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:48.505300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:48.505300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:48.505300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:48.505300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 310/602] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:48.506623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7122be4-4efa-4923-bf3e-568c90ff43bc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:48.517465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09512bd0-0fc3-402d-9dd0-710a8b78e4a9 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:48.561217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba74fc8-175a-432c-ab43-63edf9d1aab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:48.562274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba74fc8-175a-432c-ab43-63edf9d1aab1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:48.562968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba74fc8-175a-432c-ab43-63edf9d1aab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:48.563216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba74fc8-175a-432c-ab43-63edf9d1aab1 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:48.564999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba74fc8-175a-432c-ab43-63edf9d1aab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:48.565610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba74fc8-175a-432c-ab43-63edf9d1aab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:48.568156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba74fc8-175a-432c-ab43-63edf9d1aab1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:48.594707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 293/603] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:48.594707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba74fc8-175a-432c-ab43-63edf9d1aab1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:48.665933 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7122be4-4efa-4923-bf3e-568c90ff43bc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:48.715805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3a941-58ba-406f-9215-ed6482caca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:48.716950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3a941-58ba-406f-9215-ed6482caca27 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:48.717478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3a941-58ba-406f-9215-ed6482caca27 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:48.718091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3a941-58ba-406f-9215-ed6482caca27 None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/1c4da5b2130d4fa990adb72f0f6ec2fe/roles/50ceaff2140f4e05a56877fee2f609e4` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:48.721467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:48.732782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba74fc8-175a-432c-ab43-63edf9d1aab1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:48.747321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3a941-58ba-406f-9215-ed6482caca27 None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=1c4da5b2130d4fa990adb72f0f6ec2fe, role_id=50ceaff2140f4e05a56877fee2f609e4)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:48.749896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3a941-58ba-406f-9215-ed6482caca27 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:48.781137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bc168-fb49-4bdd-9a9f-cbf2bb0856f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:48.784021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bc168-fb49-4bdd-9a9f-cbf2bb0856f5 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:48.784021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bc168-fb49-4bdd-9a9f-cbf2bb0856f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:48.784021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bc168-fb49-4bdd-9a9f-cbf2bb0856f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:48.794962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bc168-fb49-4bdd-9a9f-cbf2bb0856f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-dd7bc168-fb49-4bdd-9a9f-cbf2bb0856f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:48.800541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:48.804137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 294/604] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:48.805172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bc168-fb49-4bdd-9a9f-cbf2bb0856f5 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:48.966897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 311/605] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:06:48 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/1c4da5b2130d4fa990adb72f0f6ec2fe/roles/50ceaff2140f4e05a56877fee2f609e4 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:06:48.966897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3a941-58ba-406f-9215-ed6482caca27 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:48.991768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bc168-fb49-4bdd-9a9f-cbf2bb0856f5 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:49.022334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:49.048671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a4b00-85cc-401d-8f40-d5f5a3ddd4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:49.051791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a4b00-85cc-401d-8f40-d5f5a3ddd4cc None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:49.052408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a4b00-85cc-401d-8f40-d5f5a3ddd4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:49.053003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a4b00-85cc-401d-8f40-d5f5a3ddd4cc None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:49.054439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a4b00-85cc-401d-8f40-d5f5a3ddd4cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:49.055120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a4b00-85cc-401d-8f40-d5f5a3ddd4cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:49.057802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a4b00-85cc-401d-8f40-d5f5a3ddd4cc None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:49.088264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 295/606] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:49.089921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a4b00-85cc-401d-8f40-d5f5a3ddd4cc None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:49.122255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:49.167596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3a941-58ba-406f-9215-ed6482caca27 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:49.233306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689c54b-2475-446b-9b67-16b930ea36be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:49.234426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689c54b-2475-446b-9b67-16b930ea36be None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:49.234869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689c54b-2475-446b-9b67-16b930ea36be None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:49.235392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d689c54b-2475-446b-9b67-16b930ea36be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:49.247246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689c54b-2475-446b-9b67-16b930ea36be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:49.249864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d689c54b-2475-446b-9b67-16b930ea36be None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-d689c54b-2475-446b-9b67-16b930ea36be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:49.273686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:49.274851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:49.276000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:49.276000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:49.276000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:49.276000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:49.276000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:49.276000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:49.276000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:49.276000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:49.279865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 312/607] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:49.279865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689c54b-2475-446b-9b67-16b930ea36be None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:49.286076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a4b00-85cc-401d-8f40-d5f5a3ddd4cc None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:49.333127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f6527-3f90-40e4-a4c6-51953a8a1d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:49.333846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f6527-3f90-40e4-a4c6-51953a8a1d3c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:49.334183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f6527-3f90-40e4-a4c6-51953a8a1d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:49.334514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f6527-3f90-40e4-a4c6-51953a8a1d3c None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:49.335203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f6527-3f90-40e4-a4c6-51953a8a1d3c None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:49.335513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f6527-3f90-40e4-a4c6-51953a8a1d3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:49.336970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f6527-3f90-40e4-a4c6-51953a8a1d3c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:49.358654 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 296/608] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:49.361243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f6527-3f90-40e4-a4c6-51953a8a1d3c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:49.413458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:49.440804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d689c54b-2475-446b-9b67-16b930ea36be None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:49.488742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:49.517280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d5f234-0a48-4338-9b82-9fad3604ac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:49.517280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5f234-0a48-4338-9b82-9fad3604ac0e None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:49.517280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5f234-0a48-4338-9b82-9fad3604ac0e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:49.517280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d5f234-0a48-4338-9b82-9fad3604ac0e None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/6f10c258fe5f45ccab1669fa32ed6029/roles/50ceaff2140f4e05a56877fee2f609e4` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:49.536448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5f234-0a48-4338-9b82-9fad3604ac0e None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=6f10c258fe5f45ccab1669fa32ed6029, role_id=50ceaff2140f4e05a56877fee2f609e4)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:49.538423 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d5f234-0a48-4338-9b82-9fad3604ac0e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:49.556699 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f6527-3f90-40e4-a4c6-51953a8a1d3c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:49.607264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb4efa-e237-4c67-975f-cd106923f6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:49.608196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb4efa-e237-4c67-975f-cd106923f6bf None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:49.608624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb4efa-e237-4c67-975f-cd106923f6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:49.609086 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb4efa-e237-4c67-975f-cd106923f6bf None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:49.609976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb4efa-e237-4c67-975f-cd106923f6bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:49.610473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb4efa-e237-4c67-975f-cd106923f6bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:49.612226 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb4efa-e237-4c67-975f-cd106923f6bf None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:49.627576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 297/609] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:49.628630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb4efa-e237-4c67-975f-cd106923f6bf None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:49.720453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 313/610] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:06:49 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/6f10c258fe5f45ccab1669fa32ed6029/roles/50ceaff2140f4e05a56877fee2f609e4 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:06:49.722194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57475c8e-b61d-4d3f-8fb0-91df92c84e5a None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:49.722552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57475c8e-b61d-4d3f-8fb0-91df92c84e5a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:49.722910 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57475c8e-b61d-4d3f-8fb0-91df92c84e5a None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:49.724730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 314/611] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:49.725463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57475c8e-b61d-4d3f-8fb0-91df92c84e5a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:49.761247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb4efa-e237-4c67-975f-cd106923f6bf None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:49.810912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:49.819246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2eb182-d4ff-42d4-8a8b-657a17a296cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:49.820454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2eb182-d4ff-42d4-8a8b-657a17a296cf None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:49.821689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2eb182-d4ff-42d4-8a8b-657a17a296cf None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:49.822315 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2eb182-d4ff-42d4-8a8b-657a17a296cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:49.843465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2eb182-d4ff-42d4-8a8b-657a17a296cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:49.845456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2eb182-d4ff-42d4-8a8b-657a17a296cf None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:49.884022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57475c8e-b61d-4d3f-8fb0-91df92c84e5a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-0a2eb182-d4ff-42d4-8a8b-657a17a296cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:49.885448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:49.886495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:49.887482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:49.887482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:49.887482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:49.890333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 298/612] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:49.893313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbdddb-8d8e-4d27-ba8a-1dc7b617e38b None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:49.893975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbdddb-8d8e-4d27-ba8a-1dc7b617e38b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:49.894573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbdddb-8d8e-4d27-ba8a-1dc7b617e38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:49.922593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-9dbbdddb-8d8e-4d27-ba8a-1dc7b617e38b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:49.925035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9dbbdddb-8d8e-4d27-ba8a-1dc7b617e38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:49.928927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:49.935475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c64689-f7bc-496d-8a10-bdac04814d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:49.936359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c64689-f7bc-496d-8a10-bdac04814d2a None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:49.936754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c64689-f7bc-496d-8a10-bdac04814d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:49.937243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c64689-f7bc-496d-8a10-bdac04814d2a None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/3b0a7b4f607d4753bf142e1420bcd987/roles/50ceaff2140f4e05a56877fee2f609e4` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:49.955765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c64689-f7bc-496d-8a10-bdac04814d2a None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=3b0a7b4f607d4753bf142e1420bcd987, role_id=50ceaff2140f4e05a56877fee2f609e4)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:49.957440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c64689-f7bc-496d-8a10-bdac04814d2a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:50.026083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 299/613] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:50.026083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbdddb-8d8e-4d27-ba8a-1dc7b617e38b None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:50.057086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 315/614] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:06:49 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/3b0a7b4f607d4753bf142e1420bcd987/roles/50ceaff2140f4e05a56877fee2f609e4 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:06:50.191003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbdddb-8d8e-4d27-ba8a-1dc7b617e38b None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:50.217827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:50.248406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f869fc0-e53f-47c7-b197-311747ddc1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:50.249442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f869fc0-e53f-47c7-b197-311747ddc1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:50.249831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f869fc0-e53f-47c7-b197-311747ddc1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:50.250203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f869fc0-e53f-47c7-b197-311747ddc1e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:50.251681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f869fc0-e53f-47c7-b197-311747ddc1e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:50.252148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f869fc0-e53f-47c7-b197-311747ddc1e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:50.253803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f869fc0-e53f-47c7-b197-311747ddc1e7 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:50.268524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 300/615] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:50.276280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c64689-f7bc-496d-8a10-bdac04814d2a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:50.414822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c64689-f7bc-496d-8a10-bdac04814d2a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:50.463477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c6073-b3bc-43e5-9c78-6b6d2759654e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:50.464467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c6073-b3bc-43e5-9c78-6b6d2759654e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:50.464954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c6073-b3bc-43e5-9c78-6b6d2759654e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:50.465319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c6073-b3bc-43e5-9c78-6b6d2759654e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:50.482378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c6073-b3bc-43e5-9c78-6b6d2759654e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:50.483026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f869fc0-e53f-47c7-b197-311747ddc1e7 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:50.484000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c6073-b3bc-43e5-9c78-6b6d2759654e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-dc0c6073-b3bc-43e5-9c78-6b6d2759654e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:50.498646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:50.500544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:50.502126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 316/616] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:06:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:06:50.511497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c6073-b3bc-43e5-9c78-6b6d2759654e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:50.612974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:50.645298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f869fc0-e53f-47c7-b197-311747ddc1e7 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:50.685965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4eb78-b51b-4a87-81d5-af310a0e394e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:50.687224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4eb78-b51b-4a87-81d5-af310a0e394e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:50.687781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4eb78-b51b-4a87-81d5-af310a0e394e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:50.688269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d4eb78-b51b-4a87-81d5-af310a0e394e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:50.703432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d4eb78-b51b-4a87-81d5-af310a0e394e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-98d4eb78-b51b-4a87-81d5-af310a0e394e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:50.712598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:50.714225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:50.715103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:50.718180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 301/617] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:50.724646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4eb78-b51b-4a87-81d5-af310a0e394e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:50.755463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c6073-b3bc-43e5-9c78-6b6d2759654e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:50.811506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:50.849333 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da8210-75dd-46d8-acf7-721b9f81b55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:50.852271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da8210-75dd-46d8-acf7-721b9f81b55f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:50.853047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da8210-75dd-46d8-acf7-721b9f81b55f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:50.854762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da8210-75dd-46d8-acf7-721b9f81b55f None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:50.855820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da8210-75dd-46d8-acf7-721b9f81b55f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:50.858473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da8210-75dd-46d8-acf7-721b9f81b55f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:50.861318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da8210-75dd-46d8-acf7-721b9f81b55f None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:50.907728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 317/618] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Sep 23 01:06:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:06:50.920753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da8210-75dd-46d8-acf7-721b9f81b55f None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:50.942223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d4eb78-b51b-4a87-81d5-af310a0e394e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:50.983247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bfc7d-e56f-4664-bda7-641cf4e21d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:50.984246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bfc7d-e56f-4664-bda7-641cf4e21d90 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:50.985008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bfc7d-e56f-4664-bda7-641cf4e21d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:50.985654 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bfc7d-e56f-4664-bda7-641cf4e21d90 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:50.986689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bfc7d-e56f-4664-bda7-641cf4e21d90 None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:50.987288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bfc7d-e56f-4664-bda7-641cf4e21d90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:50.988994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bfc7d-e56f-4664-bda7-641cf4e21d90 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:50.999956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 302/619] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:51.008804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bfc7d-e56f-4664-bda7-641cf4e21d90 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:51.119716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:51.126977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bfc7d-e56f-4664-bda7-641cf4e21d90 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:51.165498 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734409d-4253-4ea0-b3ad-3351bc52ef80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:51.166511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734409d-4253-4ea0-b3ad-3351bc52ef80 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:51.167001 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734409d-4253-4ea0-b3ad-3351bc52ef80 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:51.167514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734409d-4253-4ea0-b3ad-3351bc52ef80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:51.180180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734409d-4253-4ea0-b3ad-3351bc52ef80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:51.182009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734409d-4253-4ea0-b3ad-3351bc52ef80 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-d734409d-4253-4ea0-b3ad-3351bc52ef80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:51.199542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:51.201115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:51.202174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:51.202174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:51.202174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:51.205375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 303/620] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:51.211278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734409d-4253-4ea0-b3ad-3351bc52ef80 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:51.214513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:51.222504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da8210-75dd-46d8-acf7-721b9f81b55f None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:51.313757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1da814-e340-498b-ba5c-df412af7f31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:51.314315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1da814-e340-498b-ba5c-df412af7f31a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:51.314597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1da814-e340-498b-ba5c-df412af7f31a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:51.314873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1da814-e340-498b-ba5c-df412af7f31a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:51.339937 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1da814-e340-498b-ba5c-df412af7f31a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:51.341585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1da814-e340-498b-ba5c-df412af7f31a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-7c1da814-e340-498b-ba5c-df412af7f31a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:51.375995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:51.377276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:51.378433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:51.378433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:51.378433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:51.378433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 318/621] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:06:51.386872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1da814-e340-498b-ba5c-df412af7f31a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:51.410849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734409d-4253-4ea0-b3ad-3351bc52ef80 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:51.534654 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea95b2-de92-477d-bc6e-d9b03bb6a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:51.536148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea95b2-de92-477d-bc6e-d9b03bb6a210 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:51.539327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea95b2-de92-477d-bc6e-d9b03bb6a210 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:51.540110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea95b2-de92-477d-bc6e-d9b03bb6a210 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:51.541488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea95b2-de92-477d-bc6e-d9b03bb6a210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:51.544136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea95b2-de92-477d-bc6e-d9b03bb6a210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:51.548446 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea95b2-de92-477d-bc6e-d9b03bb6a210 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:51.571316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 304/622] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:51.584531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea95b2-de92-477d-bc6e-d9b03bb6a210 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:51.621629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:51.718476 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1da814-e340-498b-ba5c-df412af7f31a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:51.791354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9ff4a-7240-4005-b77a-3d83e2ace5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:51.792081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9ff4a-7240-4005-b77a-3d83e2ace5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:51.796956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9ff4a-7240-4005-b77a-3d83e2ace5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:51.797344 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9ff4a-7240-4005-b77a-3d83e2ace5a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:51.798622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9ff4a-7240-4005-b77a-3d83e2ace5a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:51.798943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9ff4a-7240-4005-b77a-3d83e2ace5a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:51.800520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9ff4a-7240-4005-b77a-3d83e2ace5a3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:51.838225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 319/623] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:51.841688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9ff4a-7240-4005-b77a-3d83e2ace5a3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:51.865164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea95b2-de92-477d-bc6e-d9b03bb6a210 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:51.871068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:51.910472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915343e9-b78e-4dfe-ac71-106cfe05442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:51.911386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915343e9-b78e-4dfe-ac71-106cfe05442e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:51.911920 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915343e9-b78e-4dfe-ac71-106cfe05442e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:51.912373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915343e9-b78e-4dfe-ac71-106cfe05442e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:51.923171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915343e9-b78e-4dfe-ac71-106cfe05442e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:51.925007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915343e9-b78e-4dfe-ac71-106cfe05442e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-915343e9-b78e-4dfe-ac71-106cfe05442e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:51.939985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:51.941176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:51.942312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:51.942312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:51.942312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:51.942312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:51.942312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:51.942312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:51.942312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:51.942312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:51.942312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 01:06:51.942312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:51.944727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 305/624] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:06:51.946246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c6b56-2cd3-4168-a471-99479d156426 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:51.946690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c6b56-2cd3-4168-a471-99479d156426 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:51.947114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c6b56-2cd3-4168-a471-99479d156426 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:51.949047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 306/625] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:51.950040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c6b56-2cd3-4168-a471-99479d156426 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:52.046335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9ff4a-7240-4005-b77a-3d83e2ace5a3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:52.112646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c46f0-402b-487b-b205-a956723c48c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:52.113553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c46f0-402b-487b-b205-a956723c48c3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:52.113998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c46f0-402b-487b-b205-a956723c48c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:52.114496 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c46f0-402b-487b-b205-a956723c48c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:52.119424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:52.123205 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c46f0-402b-487b-b205-a956723c48c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:52.126429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c46f0-402b-487b-b205-a956723c48c3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-d26c46f0-402b-487b-b205-a956723c48c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:52.141799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:52.142994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:52.144175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:52.144175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:52.144175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:52.144175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:52.144175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:52.144175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:52.144175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:52.144175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:52.146810 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 320/626] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:52.148796 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9a204-af69-42c0-83f8-0113076c4d82 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:52.149438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9a204-af69-42c0-83f8-0113076c4d82 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:52.150018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9a204-af69-42c0-83f8-0113076c4d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:52.183942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-45d9a204-af69-42c0-83f8-0113076c4d82 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:52.184815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-45d9a204-af69-42c0-83f8-0113076c4d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:52.186044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c6b56-2cd3-4168-a471-99479d156426 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:52.233585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd2296-a0a7-4523-ae57-2135cad463ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:52.236043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd2296-a0a7-4523-ae57-2135cad463ae None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:52.236712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd2296-a0a7-4523-ae57-2135cad463ae None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:52.237213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd2296-a0a7-4523-ae57-2135cad463ae None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:52.238388 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd2296-a0a7-4523-ae57-2135cad463ae None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:52.238924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd2296-a0a7-4523-ae57-2135cad463ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:52.241707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd2296-a0a7-4523-ae57-2135cad463ae None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:52.279578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 321/627] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:52.279578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9a204-af69-42c0-83f8-0113076c4d82 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:52.279578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:52.285468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 307/628] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Sep 23 01:06:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:52.292379 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd2296-a0a7-4523-ae57-2135cad463ae None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:52.401143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9a204-af69-42c0-83f8-0113076c4d82 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:52.448456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3ba9c-4516-4878-9cf6-a35b9b0009ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:52.450188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3ba9c-4516-4878-9cf6-a35b9b0009ec None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:52.450821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3ba9c-4516-4878-9cf6-a35b9b0009ec None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:52.451253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3ba9c-4516-4878-9cf6-a35b9b0009ec None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:52.452511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3ba9c-4516-4878-9cf6-a35b9b0009ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:52.452997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3ba9c-4516-4878-9cf6-a35b9b0009ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:52.456277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3ba9c-4516-4878-9cf6-a35b9b0009ec None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:52.477129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 322/629] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:52.484392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3ba9c-4516-4878-9cf6-a35b9b0009ec None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:52.574709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:52.618888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd2296-a0a7-4523-ae57-2135cad463ae None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:52.660035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3ba9c-4516-4878-9cf6-a35b9b0009ec None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:52.699463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:52.728488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56d40f-584b-4de3-9ffd-de29682d41be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:52.729699 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56d40f-584b-4de3-9ffd-de29682d41be None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:52.730319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56d40f-584b-4de3-9ffd-de29682d41be None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:52.730824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56d40f-584b-4de3-9ffd-de29682d41be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:52.732027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db1e40-5edf-4a9d-b1e2-7a3c4388b942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:52.733191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db1e40-5edf-4a9d-b1e2-7a3c4388b942 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:52.733964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db1e40-5edf-4a9d-b1e2-7a3c4388b942 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:52.734397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db1e40-5edf-4a9d-b1e2-7a3c4388b942 None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/c5a9f3eca3e34fffa7b218064e831d7b/roles/50ceaff2140f4e05a56877fee2f609e4` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:52.772219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db1e40-5edf-4a9d-b1e2-7a3c4388b942 None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=c5a9f3eca3e34fffa7b218064e831d7b, role_id=50ceaff2140f4e05a56877fee2f609e4)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:52.774352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db1e40-5edf-4a9d-b1e2-7a3c4388b942 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:52.863745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 323/630] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:06:52 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/c5a9f3eca3e34fffa7b218064e831d7b/roles/50ceaff2140f4e05a56877fee2f609e4 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:06:52.866438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db1e40-5edf-4a9d-b1e2-7a3c4388b942 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:52.916589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56d40f-584b-4de3-9ffd-de29682d41be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:52.919196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56d40f-584b-4de3-9ffd-de29682d41be None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:53.099397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db1e40-5edf-4a9d-b1e2-7a3c4388b942 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:53.129412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:53.165801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 308/631] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 874 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:53.167856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cce29-f40b-43e1-8175-e75528e3109a None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:53.168471 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cce29-f40b-43e1-8175-e75528e3109a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:53.168921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cce29-f40b-43e1-8175-e75528e3109a None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:53.171217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 309/632] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:53.172499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cce29-f40b-43e1-8175-e75528e3109a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:53.174051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451548f-dc23-4136-a863-28ddf8b12275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:53.175244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451548f-dc23-4136-a863-28ddf8b12275 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:53.175937 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451548f-dc23-4136-a863-28ddf8b12275 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:53.176515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451548f-dc23-4136-a863-28ddf8b12275 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:53.201453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451548f-dc23-4136-a863-28ddf8b12275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:53.209372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-b451548f-dc23-4136-a863-28ddf8b12275 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:53.212788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:53.213865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:53.214821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:53.215533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 324/633] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:53.217942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee1325-1703-4d48-ab92-bb23c87350ce None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:53.218412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee1325-1703-4d48-ab92-bb23c87350ce None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:53.218906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee1325-1703-4d48-ab92-bb23c87350ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:53.250449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ceee1325-1703-4d48-ab92-bb23c87350ce None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:53.253160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ceee1325-1703-4d48-ab92-bb23c87350ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:53.321250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6cce29-f40b-43e1-8175-e75528e3109a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:53.378946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 325/634] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:53.381274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b0b08-9496-40f5-bebc-1af9df967552 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:53.381644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b0b08-9496-40f5-bebc-1af9df967552 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:53.382067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b0b08-9496-40f5-bebc-1af9df967552 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:53.387772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 326/635] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:53.387772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b0b08-9496-40f5-bebc-1af9df967552 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:53.393713 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed88bb5-6cda-4e30-a116-cc2398065586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:53.394689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed88bb5-6cda-4e30-a116-cc2398065586 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:53.395019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed88bb5-6cda-4e30-a116-cc2398065586 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:53.395278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed88bb5-6cda-4e30-a116-cc2398065586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:53.407848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed88bb5-6cda-4e30-a116-cc2398065586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:53.409433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed88bb5-6cda-4e30-a116-cc2398065586 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-bed88bb5-6cda-4e30-a116-cc2398065586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:53.431355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:53.433153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:53.434823 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:53.434823 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:53.434823 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:53.437468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 310/636] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:53.439000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fad82-357e-4c6a-9dac-9a264df0db49 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:53.439289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fad82-357e-4c6a-9dac-9a264df0db49 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:53.439672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fad82-357e-4c6a-9dac-9a264df0db49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:53.462690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-024fad82-357e-4c6a-9dac-9a264df0db49 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:53.463994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-024fad82-357e-4c6a-9dac-9a264df0db49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:53.555582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 311/637] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:53.555582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fad82-357e-4c6a-9dac-9a264df0db49 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:53.576479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:53.637076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b0b08-9496-40f5-bebc-1af9df967552 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:53.652100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:53.738959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46c5a3-1f25-41a3-8c6c-116e5a54b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:53.740808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c5a3-1f25-41a3-8c6c-116e5a54b1dd None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:53.741591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c5a3-1f25-41a3-8c6c-116e5a54b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:53.742396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c5a3-1f25-41a3-8c6c-116e5a54b1dd None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:53.743797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46c5a3-1f25-41a3-8c6c-116e5a54b1dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:53.744536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46c5a3-1f25-41a3-8c6c-116e5a54b1dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:53.746975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46c5a3-1f25-41a3-8c6c-116e5a54b1dd None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:53.766637 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 327/638] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:53.776062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46c5a3-1f25-41a3-8c6c-116e5a54b1dd None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:53.866890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fad82-357e-4c6a-9dac-9a264df0db49 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:53.933087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977436fe-463d-4202-8476-9532775988fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:53.934133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977436fe-463d-4202-8476-9532775988fe None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:53.934585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977436fe-463d-4202-8476-9532775988fe None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:53.935059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977436fe-463d-4202-8476-9532775988fe None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:53.936218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977436fe-463d-4202-8476-9532775988fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:53.936738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977436fe-463d-4202-8476-9532775988fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:53.939296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977436fe-463d-4202-8476-9532775988fe None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:53.964091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 312/639] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:53.965677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977436fe-463d-4202-8476-9532775988fe None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:54.004177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46c5a3-1f25-41a3-8c6c-116e5a54b1dd None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:54.071322 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c018c2-2f53-412e-899c-56305da4a883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:54.072865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c018c2-2f53-412e-899c-56305da4a883 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:54.073644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c018c2-2f53-412e-899c-56305da4a883 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:54.074396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c018c2-2f53-412e-899c-56305da4a883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:54.095798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c018c2-2f53-412e-899c-56305da4a883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:54.098916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c018c2-2f53-412e-899c-56305da4a883 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:54.107049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-41c018c2-2f53-412e-899c-56305da4a883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:54.129095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:54.130526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:54.131999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:54.131999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:54.131999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:54.131999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 328/640] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:54.135726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c018c2-2f53-412e-899c-56305da4a883 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:54.174472 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:54.188801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977436fe-463d-4202-8476-9532775988fe None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:54.251632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8250f30-9738-49e7-adc3-626ebe6b0026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:54.253020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8250f30-9738-49e7-adc3-626ebe6b0026 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:54.253449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8250f30-9738-49e7-adc3-626ebe6b0026 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:54.253888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8250f30-9738-49e7-adc3-626ebe6b0026 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:54.272701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8250f30-9738-49e7-adc3-626ebe6b0026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-e8250f30-9738-49e7-adc3-626ebe6b0026 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:54.281353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:54.283783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:54.285430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:54.290454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 313/641] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:54.291512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8250f30-9738-49e7-adc3-626ebe6b0026 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:54.349925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c018c2-2f53-412e-899c-56305da4a883 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:54.418268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc36d9-8083-489a-b948-18b0f13fa448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:54.419371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc36d9-8083-489a-b948-18b0f13fa448 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:54.419915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc36d9-8083-489a-b948-18b0f13fa448 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:54.420461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccc36d9-8083-489a-b948-18b0f13fa448 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:54.435793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc36d9-8083-489a-b948-18b0f13fa448 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:54.440057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccc36d9-8083-489a-b948-18b0f13fa448 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-fccc36d9-8083-489a-b948-18b0f13fa448 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:54.464176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:54.466007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:54.468815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:54.468815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:54.468815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:54.468815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:54.468815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:54.468815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:54.468815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:54.468815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:54.473806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 329/642] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:54.475292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc36d9-8083-489a-b948-18b0f13fa448 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:54.477946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8250f30-9738-49e7-adc3-626ebe6b0026 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:54.526043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:54.537960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc5de4-a702-4003-881e-d2bcdf825460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:54.543218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc5de4-a702-4003-881e-d2bcdf825460 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:54.543218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc5de4-a702-4003-881e-d2bcdf825460 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:54.543218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc5de4-a702-4003-881e-d2bcdf825460 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:54.560311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc5de4-a702-4003-881e-d2bcdf825460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-b4fc5de4-a702-4003-881e-d2bcdf825460 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:54.571196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:54.572545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:54.573603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:54.578073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 314/643] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:54.580218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc5de4-a702-4003-881e-d2bcdf825460 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:54.609915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:54.651090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccc36d9-8083-489a-b948-18b0f13fa448 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:54.717145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf396c6-b90a-42f3-9113-705c6df7ecce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:54.718668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf396c6-b90a-42f3-9113-705c6df7ecce None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:54.719359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf396c6-b90a-42f3-9113-705c6df7ecce None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:54.720427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf396c6-b90a-42f3-9113-705c6df7ecce None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:54.721575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf396c6-b90a-42f3-9113-705c6df7ecce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:54.722106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf396c6-b90a-42f3-9113-705c6df7ecce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:54.724704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf396c6-b90a-42f3-9113-705c6df7ecce None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:54.729450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc5de4-a702-4003-881e-d2bcdf825460 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:54.754100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 330/644] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:54.754100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf396c6-b90a-42f3-9113-705c6df7ecce None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:54.779741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da817d-412c-4d32-a9f3-d4fe418528ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:54.780797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da817d-412c-4d32-a9f3-d4fe418528ec None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:54.781088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da817d-412c-4d32-a9f3-d4fe418528ec None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:54.781373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da817d-412c-4d32-a9f3-d4fe418528ec None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:54.782127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da817d-412c-4d32-a9f3-d4fe418528ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:54.782463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da817d-412c-4d32-a9f3-d4fe418528ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:54.783870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da817d-412c-4d32-a9f3-d4fe418528ec None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:54.802535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 315/645] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:54.802535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da817d-412c-4d32-a9f3-d4fe418528ec None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:54.893997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:54.954396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf396c6-b90a-42f3-9113-705c6df7ecce None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:54.977492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da817d-412c-4d32-a9f3-d4fe418528ec None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:54.998662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59758e8-19e8-4156-ac37-9984ba001988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:54.999542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59758e8-19e8-4156-ac37-9984ba001988 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:55.000029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59758e8-19e8-4156-ac37-9984ba001988 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:55.000439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59758e8-19e8-4156-ac37-9984ba001988 None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:55.001845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59758e8-19e8-4156-ac37-9984ba001988 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:55.002293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59758e8-19e8-4156-ac37-9984ba001988 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:55.004154 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59758e8-19e8-4156-ac37-9984ba001988 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:55.008138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:55.027672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 331/646] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:55.027672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59758e8-19e8-4156-ac37-9984ba001988 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:55.048533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8f1e8-a4d4-4cc0-b3d3-07b11a206540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:55.049176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8f1e8-a4d4-4cc0-b3d3-07b11a206540 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:55.049488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8f1e8-a4d4-4cc0-b3d3-07b11a206540 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:55.049755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8f1e8-a4d4-4cc0-b3d3-07b11a206540 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:55.050446 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8f1e8-a4d4-4cc0-b3d3-07b11a206540 None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:55.050748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8f1e8-a4d4-4cc0-b3d3-07b11a206540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:55.052368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8f1e8-a4d4-4cc0-b3d3-07b11a206540 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:55.067802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 316/647] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:55.072114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8f1e8-a4d4-4cc0-b3d3-07b11a206540 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:55.201642 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59758e8-19e8-4156-ac37-9984ba001988 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:55.256881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8f1e8-a4d4-4cc0-b3d3-07b11a206540 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:55.268761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6439109-a6e0-4058-8d6d-a8f6c34b971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:55.269874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6439109-a6e0-4058-8d6d-a8f6c34b971d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:55.270379 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6439109-a6e0-4058-8d6d-a8f6c34b971d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:55.270871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6439109-a6e0-4058-8d6d-a8f6c34b971d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:55.277033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:55.285798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6439109-a6e0-4058-8d6d-a8f6c34b971d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:55.291994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6439109-a6e0-4058-8d6d-a8f6c34b971d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:55.308881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e082bae-59e1-4253-871f-45234ea833fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:55.309809 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e082bae-59e1-4253-871f-45234ea833fa None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:55.310269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e082bae-59e1-4253-871f-45234ea833fa None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:55.310969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e082bae-59e1-4253-871f-45234ea833fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-f6439109-a6e0-4058-8d6d-a8f6c34b971d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:55.312607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:55.314240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:55.315367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:55.315367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:55.315367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:55.315367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:55.315367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:55.315367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:55.315367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:55.315367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:55.315367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:06:55.315367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:55.320703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 332/648] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Sep 23 01:06:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:55.322582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6439109-a6e0-4058-8d6d-a8f6c34b971d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:55.333714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e082bae-59e1-4253-871f-45234ea833fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:55.335310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e082bae-59e1-4253-871f-45234ea833fa None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-3e082bae-59e1-4253-871f-45234ea833fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:55.353704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:55.355355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:55.357854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:55.357854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:55.357854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:55.357854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 317/649] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:55.357854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e082bae-59e1-4253-871f-45234ea833fa None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:55.437705 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:55.516993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6439109-a6e0-4058-8d6d-a8f6c34b971d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:55.526429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e082bae-59e1-4253-871f-45234ea833fa None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:55.556553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da645782-da33-416b-bab1-6094189ad7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:55.557382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da645782-da33-416b-bab1-6094189ad7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:55.557747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da645782-da33-416b-bab1-6094189ad7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:55.558047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da645782-da33-416b-bab1-6094189ad7d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:55.558834 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da645782-da33-416b-bab1-6094189ad7d2 None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=51ae9d865bb4442490ca61b18d1f7ec7) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:55.559433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da645782-da33-416b-bab1-6094189ad7d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:55.560713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da645782-da33-416b-bab1-6094189ad7d2 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:55.577150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 333/650] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:06:55 2022] GET /identity/v3/role_assignments?user.id=51ae9d865bb4442490ca61b18d1f7ec7&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:55.580831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da645782-da33-416b-bab1-6094189ad7d2 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:55.594111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317e3a6-734d-4b9b-bc89-1fa583d31ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:55.596069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317e3a6-734d-4b9b-bc89-1fa583d31ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:55.596570 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317e3a6-734d-4b9b-bc89-1fa583d31ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:55.597061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317e3a6-734d-4b9b-bc89-1fa583d31ef6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:55.618566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317e3a6-734d-4b9b-bc89-1fa583d31ef6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:55.621463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b317e3a6-734d-4b9b-bc89-1fa583d31ef6 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-b317e3a6-734d-4b9b-bc89-1fa583d31ef6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:55.651922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:55.653091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:55.654215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:55.654215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:55.654215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:55.660261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 318/651] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:55.661953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317e3a6-734d-4b9b-bc89-1fa583d31ef6 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:55.708075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:55.763528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da645782-da33-416b-bab1-6094189ad7d2 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:55.796855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:55.826095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba126d-060e-4a75-ad37-ddcb7e75d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:55.827669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba126d-060e-4a75-ad37-ddcb7e75d67d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:55.828363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba126d-060e-4a75-ad37-ddcb7e75d67d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:55.828650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba126d-060e-4a75-ad37-ddcb7e75d67d None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:55.830103 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba126d-060e-4a75-ad37-ddcb7e75d67d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:55.830452 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba126d-060e-4a75-ad37-ddcb7e75d67d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:55.833066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba126d-060e-4a75-ad37-ddcb7e75d67d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:55.869102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 334/652] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Sep 23 01:06:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:55.869102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba126d-060e-4a75-ad37-ddcb7e75d67d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:55.869541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317e3a6-734d-4b9b-bc89-1fa583d31ef6 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:55.920419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29109dfe-8283-4d74-a0e7-fbc48e8b17df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:55.921055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29109dfe-8283-4d74-a0e7-fbc48e8b17df None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:55.921328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29109dfe-8283-4d74-a0e7-fbc48e8b17df None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:55.921657 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29109dfe-8283-4d74-a0e7-fbc48e8b17df None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:55.922353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29109dfe-8283-4d74-a0e7-fbc48e8b17df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:55.922827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29109dfe-8283-4d74-a0e7-fbc48e8b17df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:55.925252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29109dfe-8283-4d74-a0e7-fbc48e8b17df None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:55.941060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 319/653] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:55.941732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29109dfe-8283-4d74-a0e7-fbc48e8b17df None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:56.037323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba126d-060e-4a75-ad37-ddcb7e75d67d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:56.077822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:56.118472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29109dfe-8283-4d74-a0e7-fbc48e8b17df None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:56.123764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6730f7-c44a-43aa-aae7-52697ae64995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:56.124849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6730f7-c44a-43aa-aae7-52697ae64995 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:56.125353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6730f7-c44a-43aa-aae7-52697ae64995 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:56.125849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6730f7-c44a-43aa-aae7-52697ae64995 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:56.127286 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6730f7-c44a-43aa-aae7-52697ae64995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:56.127766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6730f7-c44a-43aa-aae7-52697ae64995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:56.129918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6730f7-c44a-43aa-aae7-52697ae64995 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:56.148280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 335/654] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:56.148280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6730f7-c44a-43aa-aae7-52697ae64995 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:56.168687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d6373-c7fc-4406-8786-9805dc2d612d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:56.169564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d6373-c7fc-4406-8786-9805dc2d612d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:56.170012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d6373-c7fc-4406-8786-9805dc2d612d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:56.170454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d6373-c7fc-4406-8786-9805dc2d612d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:56.207048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:56.279339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d6373-c7fc-4406-8786-9805dc2d612d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:56.281089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d6373-c7fc-4406-8786-9805dc2d612d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:56.288270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6730f7-c44a-43aa-aae7-52697ae64995 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:56.333321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5772b8-2492-45ae-b630-951ebeb5154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:56.333987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5772b8-2492-45ae-b630-951ebeb5154c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:56.334280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5772b8-2492-45ae-b630-951ebeb5154c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:56.334562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5772b8-2492-45ae-b630-951ebeb5154c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:56.343215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5772b8-2492-45ae-b630-951ebeb5154c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:56.344770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5772b8-2492-45ae-b630-951ebeb5154c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-ad5772b8-2492-45ae-b630-951ebeb5154c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:56.358373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:56.359643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:56.361515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:56.361515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:56.361515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:56.361515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:56.361515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:56.361515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:56.361515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:56.361515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:56.361515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 23 01:06:56.361515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:56.362073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 336/655] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:06:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:56.363233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753962-a645-41f2-9c52-ca8c12aaa1c5 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:56.363525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753962-a645-41f2-9c52-ca8c12aaa1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:56.363825 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33753962-a645-41f2-9c52-ca8c12aaa1c5 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:56.365563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 337/656] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:56.365928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753962-a645-41f2-9c52-ca8c12aaa1c5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:56.388131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:56.425153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 320/657] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:56.426887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d6373-c7fc-4406-8786-9805dc2d612d None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:56.506338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:56.511686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33753962-a645-41f2-9c52-ca8c12aaa1c5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:56.555602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081af1f-21ba-4322-bef2-6a50b861ac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:56.556703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081af1f-21ba-4322-bef2-6a50b861ac50 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:56.557108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081af1f-21ba-4322-bef2-6a50b861ac50 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:56.557466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081af1f-21ba-4322-bef2-6a50b861ac50 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:56.567149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081af1f-21ba-4322-bef2-6a50b861ac50 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:56.569054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1081af1f-21ba-4322-bef2-6a50b861ac50 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:56.581299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d6373-c7fc-4406-8786-9805dc2d612d None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-1081af1f-21ba-4322-bef2-6a50b861ac50 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:56.583706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:56.592826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 338/658] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fdaec-2408-45e4-a7fc-f6bef5a08cb7 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:56.594904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fdaec-2408-45e4-a7fc-f6bef5a08cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:56.596550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fdaec-2408-45e4-a7fc-f6bef5a08cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:56.614514 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b81fdaec-2408-45e4-a7fc-f6bef5a08cb7 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:56.616114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b81fdaec-2408-45e4-a7fc-f6bef5a08cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:56.643270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd46dd-cfed-40b2-b7e6-e831c98c5226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:56.643979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd46dd-cfed-40b2-b7e6-e831c98c5226 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:56.644262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd46dd-cfed-40b2-b7e6-e831c98c5226 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:56.644595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd46dd-cfed-40b2-b7e6-e831c98c5226 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:56.645381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd46dd-cfed-40b2-b7e6-e831c98c5226 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:56.645728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd46dd-cfed-40b2-b7e6-e831c98c5226 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:56.647577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd46dd-cfed-40b2-b7e6-e831c98c5226 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:56.676698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 321/659] 10.222.0.64 () {56 vars in 1137 bytes} [Fri Sep 23 01:06:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:56.676698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd46dd-cfed-40b2-b7e6-e831c98c5226 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:56.717941 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 339/660] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:56.719021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fdaec-2408-45e4-a7fc-f6bef5a08cb7 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:56.720982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:56.820589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd46dd-cfed-40b2-b7e6-e831c98c5226 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:56.866965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a75424-3084-4026-bcd4-7ca3e900f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:56.867891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a75424-3084-4026-bcd4-7ca3e900f6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:56.868254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a75424-3084-4026-bcd4-7ca3e900f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:56.868608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a75424-3084-4026-bcd4-7ca3e900f6f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:56.883776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a75424-3084-4026-bcd4-7ca3e900f6f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:56.885998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a75424-3084-4026-bcd4-7ca3e900f6f2 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-b4a75424-3084-4026-bcd4-7ca3e900f6f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:56.905305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:56.907441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:56.909477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:56.909477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:56.909477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:56.909477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 322/661] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:56.910483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a75424-3084-4026-bcd4-7ca3e900f6f2 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:56.916891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:56.938301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fdaec-2408-45e4-a7fc-f6bef5a08cb7 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:56.991057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d01ce2-a00f-47de-8bc9-9c787dbe02db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:56.993080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d01ce2-a00f-47de-8bc9-9c787dbe02db None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:56.993080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d01ce2-a00f-47de-8bc9-9c787dbe02db None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:56.993080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d01ce2-a00f-47de-8bc9-9c787dbe02db None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:56.993520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d01ce2-a00f-47de-8bc9-9c787dbe02db None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:56.993849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d01ce2-a00f-47de-8bc9-9c787dbe02db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:56.995481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d01ce2-a00f-47de-8bc9-9c787dbe02db None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:57.027586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 340/662] 10.222.0.64 () {56 vars in 1139 bytes} [Fri Sep 23 01:06:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:57.027586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d01ce2-a00f-47de-8bc9-9c787dbe02db None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:57.081878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:57.108922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a75424-3084-4026-bcd4-7ca3e900f6f2 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:57.164786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d01ce2-a00f-47de-8bc9-9c787dbe02db None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:57.179019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55f23c-183a-450a-a7c9-e0eebd7d3062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:57.179693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55f23c-183a-450a-a7c9-e0eebd7d3062 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:57.179973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55f23c-183a-450a-a7c9-e0eebd7d3062 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:57.180253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55f23c-183a-450a-a7c9-e0eebd7d3062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:57.224083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600e116-7fcd-4693-aa36-bbcaee3646dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:57.225047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600e116-7fcd-4693-aa36-bbcaee3646dd None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:57.225375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600e116-7fcd-4693-aa36-bbcaee3646dd None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:57.225623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600e116-7fcd-4693-aa36-bbcaee3646dd None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:57.226272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600e116-7fcd-4693-aa36-bbcaee3646dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:57.226581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600e116-7fcd-4693-aa36-bbcaee3646dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:57.228455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600e116-7fcd-4693-aa36-bbcaee3646dd None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:57.249318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 341/663] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:57.251779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600e116-7fcd-4693-aa36-bbcaee3646dd None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:57.267946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:57.316270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55f23c-183a-450a-a7c9-e0eebd7d3062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:57.318192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55f23c-183a-450a-a7c9-e0eebd7d3062 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:57.412652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600e116-7fcd-4693-aa36-bbcaee3646dd None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:57.459670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:57.464720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834250c-8fe0-457c-a9f2-38f07410327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:57.465524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834250c-8fe0-457c-a9f2-38f07410327c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:57.465934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834250c-8fe0-457c-a9f2-38f07410327c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:57.466127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834250c-8fe0-457c-a9f2-38f07410327c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:57.494972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 323/664] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:57.496994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55f23c-183a-450a-a7c9-e0eebd7d3062 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:57.552974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834250c-8fe0-457c-a9f2-38f07410327c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:57.554406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834250c-8fe0-457c-a9f2-38f07410327c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:57.565424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:57.671046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55f23c-183a-450a-a7c9-e0eebd7d3062 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:57.734424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 342/665] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:06:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:06:57.736044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834250c-8fe0-457c-a9f2-38f07410327c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:57.745305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f06a2-bba1-4fb9-8688-8c48fe0c2a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:57.746075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f06a2-bba1-4fb9-8688-8c48fe0c2a16 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:57.746359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f06a2-bba1-4fb9-8688-8c48fe0c2a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:57.746631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f06a2-bba1-4fb9-8688-8c48fe0c2a16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:57.762465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f06a2-bba1-4fb9-8688-8c48fe0c2a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-a21f06a2-bba1-4fb9-8688-8c48fe0c2a16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:06:57.768592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:57.770420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 324/666] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:06:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:57.771396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f06a2-bba1-4fb9-8688-8c48fe0c2a16 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:57.851862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:57.894675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834250c-8fe0-457c-a9f2-38f07410327c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:57.905391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:57.941202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f1692-9c72-47f2-a40f-1bbbb1c2cb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:57.941903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f1692-9c72-47f2-a40f-1bbbb1c2cb3b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:57.942178 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f1692-9c72-47f2-a40f-1bbbb1c2cb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:57.942445 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f1692-9c72-47f2-a40f-1bbbb1c2cb3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:57.952998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f06a2-bba1-4fb9-8688-8c48fe0c2a16 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:57.958794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f1692-9c72-47f2-a40f-1bbbb1c2cb3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:57.961630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f1692-9c72-47f2-a40f-1bbbb1c2cb3b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-a95f1692-9c72-47f2-a40f-1bbbb1c2cb3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:57.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:57.989635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:57.990616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:57.990616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:57.990616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:57.990616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:57.990616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:57.990616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:57.990616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:57.990616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:57.997905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 343/667] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:57.999530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f1692-9c72-47f2-a40f-1bbbb1c2cb3b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:58.003863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aa8a5-04ba-4834-a409-6d8dd76b90e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:58.004660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aa8a5-04ba-4834-a409-6d8dd76b90e7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:58.004946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aa8a5-04ba-4834-a409-6d8dd76b90e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:58.005224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aa8a5-04ba-4834-a409-6d8dd76b90e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:58.018146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aa8a5-04ba-4834-a409-6d8dd76b90e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:58.020091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aa8a5-04ba-4834-a409-6d8dd76b90e7 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-2e0aa8a5-04ba-4834-a409-6d8dd76b90e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:58.037452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:58.038582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:58.039771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:58.039771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:58.039771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:58.039771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 325/668] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:58.040059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aa8a5-04ba-4834-a409-6d8dd76b90e7 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:58.158078 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f1692-9c72-47f2-a40f-1bbbb1c2cb3b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:58.186171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:58.191862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aa8a5-04ba-4834-a409-6d8dd76b90e7 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:58.207047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecad53-fc90-4b25-a27c-0347419388f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:58.208018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecad53-fc90-4b25-a27c-0347419388f4 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:58.208328 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecad53-fc90-4b25-a27c-0347419388f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:58.208710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecad53-fc90-4b25-a27c-0347419388f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:58.221624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecad53-fc90-4b25-a27c-0347419388f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:58.223232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecad53-fc90-4b25-a27c-0347419388f4 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:58.240972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e737ec-4094-4ceb-b049-01559a0334a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-39ecad53-fc90-4b25-a27c-0347419388f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:58.241254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:58.242976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:58.244777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:58.244777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:58.244777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:58.245104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e737ec-4094-4ceb-b049-01559a0334a0 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:58.245104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e737ec-4094-4ceb-b049-01559a0334a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:58.245349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 344/669] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:58.245349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecad53-fc90-4b25-a27c-0347419388f4 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:58.248874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e737ec-4094-4ceb-b049-01559a0334a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:58.250012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e737ec-4094-4ceb-b049-01559a0334a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:58.250452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e737ec-4094-4ceb-b049-01559a0334a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:58.253143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e737ec-4094-4ceb-b049-01559a0334a0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:58.263922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 326/670] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:06:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:58.264928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e737ec-4094-4ceb-b049-01559a0334a0 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:58.276762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:58.398985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecad53-fc90-4b25-a27c-0347419388f4 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:58.402897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e737ec-4094-4ceb-b049-01559a0334a0 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:58.442216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a71a78-bf2e-4c97-a04b-2c074e6f3c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:58.443263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a71a78-bf2e-4c97-a04b-2c074e6f3c1d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:58.444077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a71a78-bf2e-4c97-a04b-2c074e6f3c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:58.444585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a71a78-bf2e-4c97-a04b-2c074e6f3c1d None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:58.445953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a71a78-bf2e-4c97-a04b-2c074e6f3c1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:58.446523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a71a78-bf2e-4c97-a04b-2c074e6f3c1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:58.449041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8fb67-2e8e-492f-8de8-3c217b570a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:58.449312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a71a78-bf2e-4c97-a04b-2c074e6f3c1d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:58.449826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8fb67-2e8e-492f-8de8-3c217b570a5e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:58.450203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8fb67-2e8e-492f-8de8-3c217b570a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:58.450560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8fb67-2e8e-492f-8de8-3c217b570a5e None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:58.451420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8fb67-2e8e-492f-8de8-3c217b570a5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:58.451852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8fb67-2e8e-492f-8de8-3c217b570a5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:58.453540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8fb67-2e8e-492f-8de8-3c217b570a5e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:58.468770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 327/671] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:58.469715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8fb67-2e8e-492f-8de8-3c217b570a5e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:58.478389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 345/672] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:58.478389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a71a78-bf2e-4c97-a04b-2c074e6f3c1d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:58.495965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:58.596437 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:58.597244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8fb67-2e8e-492f-8de8-3c217b570a5e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:58.630719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a71a78-bf2e-4c97-a04b-2c074e6f3c1d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:58.639260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df256988-de83-4655-9c4b-80c05e79ca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:58.639914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df256988-de83-4655-9c4b-80c05e79ca07 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:58.640293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df256988-de83-4655-9c4b-80c05e79ca07 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:58.640613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df256988-de83-4655-9c4b-80c05e79ca07 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:58.641361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df256988-de83-4655-9c4b-80c05e79ca07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:58.641672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df256988-de83-4655-9c4b-80c05e79ca07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:58.643082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df256988-de83-4655-9c4b-80c05e79ca07 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:58.656257 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 328/673] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:58.657658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa797-9671-448d-8087-4ed32ea0ae58 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:58.657952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa797-9671-448d-8087-4ed32ea0ae58 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:58.658386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892aa797-9671-448d-8087-4ed32ea0ae58 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:58.659771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 329/674] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:06:58.660518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aa797-9671-448d-8087-4ed32ea0ae58 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:58.698172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527d3d1-8bcb-4d9e-9f69-dc55b1f67215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:58.699286 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527d3d1-8bcb-4d9e-9f69-dc55b1f67215 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:58.699793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527d3d1-8bcb-4d9e-9f69-dc55b1f67215 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:58.700293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527d3d1-8bcb-4d9e-9f69-dc55b1f67215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:58.718419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527d3d1-8bcb-4d9e-9f69-dc55b1f67215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:58.721299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527d3d1-8bcb-4d9e-9f69-dc55b1f67215 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-9527d3d1-8bcb-4d9e-9f69-dc55b1f67215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:06:58.752464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:58.753780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:06:58.755189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:06:58.755189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:06:58.755189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:58.755189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 346/675] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:58.756458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527d3d1-8bcb-4d9e-9f69-dc55b1f67215 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:58.758621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:58.785011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892aa797-9671-448d-8087-4ed32ea0ae58 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:58.837447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d67e5-35a4-44cb-a612-defc51a798e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:58.838100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d67e5-35a4-44cb-a612-defc51a798e6 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:58.838380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d67e5-35a4-44cb-a612-defc51a798e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:58.838655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d67e5-35a4-44cb-a612-defc51a798e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:58.847411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d67e5-35a4-44cb-a612-defc51a798e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:58.849016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d67e5-35a4-44cb-a612-defc51a798e6 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-982d67e5-35a4-44cb-a612-defc51a798e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:58.863736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:58.865484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:58.866802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:58.866802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:58.866802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:58.866802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:58.866802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:58.866802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:58.866802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:58.866802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:58.866802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 330/676] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:58.866802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d67e5-35a4-44cb-a612-defc51a798e6 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:58.921931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527d3d1-8bcb-4d9e-9f69-dc55b1f67215 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:58.970780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:58.979581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b17cc-b418-488c-9e6f-456354428cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:58.980571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b17cc-b418-488c-9e6f-456354428cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:58.982864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b17cc-b418-488c-9e6f-456354428cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:58.982864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b17cc-b418-488c-9e6f-456354428cd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:58.982864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b17cc-b418-488c-9e6f-456354428cd0 None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=1c4da5b2130d4fa990adb72f0f6ec2fe) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:58.983521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b17cc-b418-488c-9e6f-456354428cd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:58.986270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b17cc-b418-488c-9e6f-456354428cd0 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:59.009894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 347/677] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:06:58 2022] GET /identity/v3/role_assignments?user.id=1c4da5b2130d4fa990adb72f0f6ec2fe&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:59.009894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cca7d3-eb5d-44a4-a265-c0e9513044f0 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:59.009894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cca7d3-eb5d-44a4-a265-c0e9513044f0 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:59.009894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cca7d3-eb5d-44a4-a265-c0e9513044f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:59.014330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d67e5-35a4-44cb-a612-defc51a798e6 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:59.036273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-17cca7d3-eb5d-44a4-a265-c0e9513044f0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:06:59.038284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-17cca7d3-eb5d-44a4-a265-c0e9513044f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:06:59.072619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9ce1e-b688-4944-924e-59ee217cd5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:59.073435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9ce1e-b688-4944-924e-59ee217cd5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:59.073891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9ce1e-b688-4944-924e-59ee217cd5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:59.074352 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9ce1e-b688-4944-924e-59ee217cd5e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:59.086243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9ce1e-b688-4944-924e-59ee217cd5e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:59.088930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9ce1e-b688-4944-924e-59ee217cd5e8 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:59.093879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-ffe9ce1e-b688-4944-924e-59ee217cd5e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:06:59.112188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:59.113637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:59.120222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:06:59.120222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:06:59.120222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:06:59.120222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:06:59.120222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:06:59.120222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:06:59.120222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:06:59.120222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:06:59.120222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 331/678] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:06:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:59.120222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9ce1e-b688-4944-924e-59ee217cd5e8 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:59.179641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 348/679] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:06:59.179641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cca7d3-eb5d-44a4-a265-c0e9513044f0 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:59.309374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9ce1e-b688-4944-924e-59ee217cd5e8 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:59.370442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2834f-5a88-46de-8a8d-25b8588396a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:59.372123 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2834f-5a88-46de-8a8d-25b8588396a6 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:59.372846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2834f-5a88-46de-8a8d-25b8588396a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:59.373882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2834f-5a88-46de-8a8d-25b8588396a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:59.375939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2834f-5a88-46de-8a8d-25b8588396a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:59.375939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2834f-5a88-46de-8a8d-25b8588396a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:59.379421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2834f-5a88-46de-8a8d-25b8588396a6 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:59.395784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cca7d3-eb5d-44a4-a265-c0e9513044f0 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:59.403625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 332/680] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:59.406574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2834f-5a88-46de-8a8d-25b8588396a6 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:59.441180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:59.465202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0f3d-4ed8-4ff5-bfa4-bea454f311af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:59.466266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0f3d-4ed8-4ff5-bfa4-bea454f311af None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:59.466753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0f3d-4ed8-4ff5-bfa4-bea454f311af None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:59.467145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0f3d-4ed8-4ff5-bfa4-bea454f311af None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:59.468823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b0f3d-4ed8-4ff5-bfa4-bea454f311af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:59.471326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b0f3d-4ed8-4ff5-bfa4-bea454f311af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:59.474352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b0f3d-4ed8-4ff5-bfa4-bea454f311af None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:59.500046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 349/681] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:59.502797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0f3d-4ed8-4ff5-bfa4-bea454f311af None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:59.521592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:59.621909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2834f-5a88-46de-8a8d-25b8588396a6 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:59.678386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0f3d-4ed8-4ff5-bfa4-bea454f311af None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:59.691312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d959fd7-a97d-4a12-859e-dee7d503674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:59.692415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d959fd7-a97d-4a12-859e-dee7d503674a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:59.692922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d959fd7-a97d-4a12-859e-dee7d503674a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:59.693432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d959fd7-a97d-4a12-859e-dee7d503674a None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:59.695050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d959fd7-a97d-4a12-859e-dee7d503674a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:59.695639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d959fd7-a97d-4a12-859e-dee7d503674a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:59.698596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d959fd7-a97d-4a12-859e-dee7d503674a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:59.724118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 333/682] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:06:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:06:59.725671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d959fd7-a97d-4a12-859e-dee7d503674a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:59.736674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13ab71-7672-4df0-9aa2-596204f9e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:59.737582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ab71-7672-4df0-9aa2-596204f9e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:59.738060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ab71-7672-4df0-9aa2-596204f9e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:59.738738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13ab71-7672-4df0-9aa2-596204f9e3f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:59.749500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13ab71-7672-4df0-9aa2-596204f9e3f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:59.751260 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13ab71-7672-4df0-9aa2-596204f9e3f5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-3e13ab71-7672-4df0-9aa2-596204f9e3f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:06:59.765697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:06:59.767771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 350/683] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:06:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:06:59.769668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13ab71-7672-4df0-9aa2-596204f9e3f5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:06:59.845234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:59.917392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d959fd7-a97d-4a12-859e-dee7d503674a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:59.928697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13ab71-7672-4df0-9aa2-596204f9e3f5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:06:59.950491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:06:59.977463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d3bf4-568b-4ec4-bc8b-a27ea1aa1f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:59.978420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d3bf4-568b-4ec4-bc8b-a27ea1aa1f09 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:59.978838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d3bf4-568b-4ec4-bc8b-a27ea1aa1f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:59.979519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d3bf4-568b-4ec4-bc8b-a27ea1aa1f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:59.980423 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d3bf4-568b-4ec4-bc8b-a27ea1aa1f09 None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=6f10c258fe5f45ccab1669fa32ed6029) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:06:59.981476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d3bf4-568b-4ec4-bc8b-a27ea1aa1f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:59.983919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d3bf4-568b-4ec4-bc8b-a27ea1aa1f09 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:06:59.984693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32f456-731e-4705-9053-cfabc3bb3321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:06:59.986032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32f456-731e-4705-9053-cfabc3bb3321 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:06:59.986317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32f456-731e-4705-9053-cfabc3bb3321 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:06:59.986706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32f456-731e-4705-9053-cfabc3bb3321 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:06:59.987314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32f456-731e-4705-9053-cfabc3bb3321 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:06:59.989036 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32f456-731e-4705-9053-cfabc3bb3321 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:00.003804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 334/684] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:06:59 2022] GET /identity/v3/role_assignments?user.id=6f10c258fe5f45ccab1669fa32ed6029&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:00.003804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d3bf4-568b-4ec4-bc8b-a27ea1aa1f09 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-8e32f456-731e-4705-9053-cfabc3bb3321 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:00.014951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:00.016945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:00.018179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 351/685] 10.222.0.64 () {56 vars in 1129 bytes} [Fri Sep 23 01:06:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:00.020544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32f456-731e-4705-9053-cfabc3bb3321 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:00.185863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d3bf4-568b-4ec4-bc8b-a27ea1aa1f09 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:00.250442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32f456-731e-4705-9053-cfabc3bb3321 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:00.262378 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13570d-4f23-4dd7-b73c-c96b331caa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:00.264390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13570d-4f23-4dd7-b73c-c96b331caa9d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:00.265253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13570d-4f23-4dd7-b73c-c96b331caa9d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:00.266058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13570d-4f23-4dd7-b73c-c96b331caa9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:00.269959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:00.272714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13570d-4f23-4dd7-b73c-c96b331caa9d None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=3b0a7b4f607d4753bf142e1420bcd987) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:00.273625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13570d-4f23-4dd7-b73c-c96b331caa9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:00.276796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13570d-4f23-4dd7-b73c-c96b331caa9d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:00.296012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 335/686] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:06:59 2022] GET /identity/v3/role_assignments?user.id=3b0a7b4f607d4753bf142e1420bcd987&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:00.297877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13570d-4f23-4dd7-b73c-c96b331caa9d None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:00.325323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9444d5-1dc2-4786-a59e-38f4fb37f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:00.325973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9444d5-1dc2-4786-a59e-38f4fb37f564 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:00.326242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9444d5-1dc2-4786-a59e-38f4fb37f564 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:00.326523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9444d5-1dc2-4786-a59e-38f4fb37f564 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:00.327422 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9444d5-1dc2-4786-a59e-38f4fb37f564 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:00.327805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9444d5-1dc2-4786-a59e-38f4fb37f564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:00.329460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9444d5-1dc2-4786-a59e-38f4fb37f564 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:00.370499 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 352/687] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Sep 23 01:07:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:00.380653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9444d5-1dc2-4786-a59e-38f4fb37f564 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:00.419976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:00.514342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9444d5-1dc2-4786-a59e-38f4fb37f564 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:00.554516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5532fc-187e-4e05-a400-d26e1419415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:00.555155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5532fc-187e-4e05-a400-d26e1419415a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:00.555429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5532fc-187e-4e05-a400-d26e1419415a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:00.555734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5532fc-187e-4e05-a400-d26e1419415a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:00.567297 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13570d-4f23-4dd7-b73c-c96b331caa9d None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:00.624108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:00.637273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee8949-5c6c-46f7-83c9-edc0416a3596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:00.637922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee8949-5c6c-46f7-83c9-edc0416a3596 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:00.639425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee8949-5c6c-46f7-83c9-edc0416a3596 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:00.639425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee8949-5c6c-46f7-83c9-edc0416a3596 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:00.639425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee8949-5c6c-46f7-83c9-edc0416a3596 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:00.639956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee8949-5c6c-46f7-83c9-edc0416a3596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:00.641268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee8949-5c6c-46f7-83c9-edc0416a3596 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:00.649368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 336/688] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Sep 23 01:07:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:00.666248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee8949-5c6c-46f7-83c9-edc0416a3596 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:00.724226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5532fc-187e-4e05-a400-d26e1419415a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:00.728225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5532fc-187e-4e05-a400-d26e1419415a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:00.904689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee8949-5c6c-46f7-83c9-edc0416a3596 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:00.949038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:00.986900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 353/689] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:07:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 01:07:00.993552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5532fc-187e-4e05-a400-d26e1419415a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:01.042324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32c0fa-5b58-4ec6-80d5-cd04bf381515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:01.043491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32c0fa-5b58-4ec6-80d5-cd04bf381515 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:01.044050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32c0fa-5b58-4ec6-80d5-cd04bf381515 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:01.054622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32c0fa-5b58-4ec6-80d5-cd04bf381515 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:01.054622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32c0fa-5b58-4ec6-80d5-cd04bf381515 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:01.054622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32c0fa-5b58-4ec6-80d5-cd04bf381515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:01.056187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32c0fa-5b58-4ec6-80d5-cd04bf381515 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:01.067843 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 337/690] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Sep 23 01:07:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:01.166413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:01.246628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5532fc-187e-4e05-a400-d26e1419415a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:01.340902 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e04e2-b413-4949-b7be-54f9e645a31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:01.342025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e04e2-b413-4949-b7be-54f9e645a31a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:01.342520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e04e2-b413-4949-b7be-54f9e645a31a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:01.343005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e04e2-b413-4949-b7be-54f9e645a31a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:01.368276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e04e2-b413-4949-b7be-54f9e645a31a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:01.370976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e04e2-b413-4949-b7be-54f9e645a31a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-025e04e2-b413-4949-b7be-54f9e645a31a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:01.404153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:01.405223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:01.406142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:01.406142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:01.406142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:01.407617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 354/691] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:01.416923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32c0fa-5b58-4ec6-80d5-cd04bf381515 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:01.642454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32c0fa-5b58-4ec6-80d5-cd04bf381515 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:01.726399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19541c-330f-46d4-8303-fdf70c169988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:01.727357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19541c-330f-46d4-8303-fdf70c169988 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:01.727889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19541c-330f-46d4-8303-fdf70c169988 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:01.728465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19541c-330f-46d4-8303-fdf70c169988 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:01.729697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19541c-330f-46d4-8303-fdf70c169988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:01.730266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19541c-330f-46d4-8303-fdf70c169988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:01.732838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19541c-330f-46d4-8303-fdf70c169988 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:01.764697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 338/692] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:01.774056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e04e2-b413-4949-b7be-54f9e645a31a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:01.963206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e04e2-b413-4949-b7be-54f9e645a31a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:01.985567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:02.007785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccc7ca-e7f1-4e4c-936a-27a4a3ae37bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:02.008552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccc7ca-e7f1-4e4c-936a-27a4a3ae37bb None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:02.008828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccc7ca-e7f1-4e4c-936a-27a4a3ae37bb None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:02.009113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccc7ca-e7f1-4e4c-936a-27a4a3ae37bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:02.017787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccc7ca-e7f1-4e4c-936a-27a4a3ae37bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:02.019485 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ccc7ca-e7f1-4e4c-936a-27a4a3ae37bb None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-e1ccc7ca-e7f1-4e4c-936a-27a4a3ae37bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:02.033892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:02.035720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:02.037516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:02.037516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:02.037516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:02.037516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:02.037516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:02.037516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:02.037516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:02.037516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:02.037516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 355/693] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:02.049342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19541c-330f-46d4-8303-fdf70c169988 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:02.165733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:02.218615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ba05a-af9d-4817-85cc-db3f8d831f57 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:02.218940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ba05a-af9d-4817-85cc-db3f8d831f57 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:02.219930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ba05a-af9d-4817-85cc-db3f8d831f57 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:02.222556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 356/694] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:02.229373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa1fa9-d817-40d1-805f-01887d571c76 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:02.229774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa1fa9-d817-40d1-805f-01887d571c76 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:02.229948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa1fa9-d817-40d1-805f-01887d571c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:02.260617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-bdfa1fa9-d817-40d1-805f-01887d571c76 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:02.262719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bdfa1fa9-d817-40d1-805f-01887d571c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:02.388702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19541c-330f-46d4-8303-fdf70c169988 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:02.394203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 357/695] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:07:02.477603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1b624-75c3-461d-91c1-86b8e323d7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:02.478321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b624-75c3-461d-91c1-86b8e323d7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:02.478604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b624-75c3-461d-91c1-86b8e323d7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:02.478879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b624-75c3-461d-91c1-86b8e323d7c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:02.479612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1b624-75c3-461d-91c1-86b8e323d7c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:02.479913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1b624-75c3-461d-91c1-86b8e323d7c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:02.481512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1b624-75c3-461d-91c1-86b8e323d7c8 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:02.501305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 339/696] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:02.507536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa1fa9-d817-40d1-805f-01887d571c76 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:02.671686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa1fa9-d817-40d1-805f-01887d571c76 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:02.691006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:02.728310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cf16f-d6e8-4d8a-9314-ef1ba385b673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:02.728975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cf16f-d6e8-4d8a-9314-ef1ba385b673 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:02.729257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cf16f-d6e8-4d8a-9314-ef1ba385b673 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:02.729531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cf16f-d6e8-4d8a-9314-ef1ba385b673 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:02.730350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cf16f-d6e8-4d8a-9314-ef1ba385b673 None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=c5a9f3eca3e34fffa7b218064e831d7b) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:02.730659 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cf16f-d6e8-4d8a-9314-ef1ba385b673 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:02.732203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10cf16f-d6e8-4d8a-9314-ef1ba385b673 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:02.746470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 358/697] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:07:02 2022] GET /identity/v3/role_assignments?user.id=c5a9f3eca3e34fffa7b218064e831d7b&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:02.998296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1b624-75c3-461d-91c1-86b8e323d7c8 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:03.183538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:03.196059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1b624-75c3-461d-91c1-86b8e323d7c8 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:03.252752 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306a07f-486a-4338-9f23-38a74029470c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:03.254255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306a07f-486a-4338-9f23-38a74029470c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:03.254991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306a07f-486a-4338-9f23-38a74029470c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:03.255790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306a07f-486a-4338-9f23-38a74029470c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:03.287274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306a07f-486a-4338-9f23-38a74029470c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-2306a07f-486a-4338-9f23-38a74029470c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:03.299701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:03.301539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:03.303186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:03.307720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 340/698] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:07:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:03.317538 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cf16f-d6e8-4d8a-9314-ef1ba385b673 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:03.457628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cf16f-d6e8-4d8a-9314-ef1ba385b673 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:03.509480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b0fc8-d8ee-4209-865e-6a06770e44d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:03.510046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b0fc8-d8ee-4209-865e-6a06770e44d3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:03.510385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b0fc8-d8ee-4209-865e-6a06770e44d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:03.510750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b0fc8-d8ee-4209-865e-6a06770e44d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:03.511653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b0fc8-d8ee-4209-865e-6a06770e44d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:03.511928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b0fc8-d8ee-4209-865e-6a06770e44d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:03.514665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b0fc8-d8ee-4209-865e-6a06770e44d3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:03.533494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 359/699] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:03.541377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306a07f-486a-4338-9f23-38a74029470c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:03.801216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306a07f-486a-4338-9f23-38a74029470c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:03.850183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f748127f-2094-452e-a3ec-11ad3c52b3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:03.851258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748127f-2094-452e-a3ec-11ad3c52b3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:03.851664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748127f-2094-452e-a3ec-11ad3c52b3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:03.852015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f748127f-2094-452e-a3ec-11ad3c52b3c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:03.872691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748127f-2094-452e-a3ec-11ad3c52b3c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:03.874207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f748127f-2094-452e-a3ec-11ad3c52b3c3 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-f748127f-2094-452e-a3ec-11ad3c52b3c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:03.901496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:03.903205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:03.904976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:03.904976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:03.904976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:03.904976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 341/700] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:03.911148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b0fc8-d8ee-4209-865e-6a06770e44d3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:03.999879 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:04.034923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04087b-eab1-49bf-806c-0694f04f9bdf None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:04.035379 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04087b-eab1-49bf-806c-0694f04f9bdf None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:04.035651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04087b-eab1-49bf-806c-0694f04f9bdf None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:04.042700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 342/701] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:04.055959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ed24f-e3b6-4409-93f6-a5c566592f14 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:04.056368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ed24f-e3b6-4409-93f6-a5c566592f14 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:04.056824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ed24f-e3b6-4409-93f6-a5c566592f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:04.121739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-097ed24f-e3b6-4409-93f6-a5c566592f14 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:04.124020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-097ed24f-e3b6-4409-93f6-a5c566592f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:04.135138 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:04.146774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b0fc8-d8ee-4209-865e-6a06770e44d3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:04.226723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b755c-bef7-4cc5-9e27-b36140a3c1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:04.227829 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b755c-bef7-4cc5-9e27-b36140a3c1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:04.228331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b755c-bef7-4cc5-9e27-b36140a3c1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:04.228896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b755c-bef7-4cc5-9e27-b36140a3c1b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:04.230115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b755c-bef7-4cc5-9e27-b36140a3c1b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:04.230664 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b755c-bef7-4cc5-9e27-b36140a3c1b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:04.233492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b755c-bef7-4cc5-9e27-b36140a3c1b0 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:04.287794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 360/702] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:04.298749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b755c-bef7-4cc5-9e27-b36140a3c1b0 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:04.384048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 343/703] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:07:04.495537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:04.509093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b755c-bef7-4cc5-9e27-b36140a3c1b0 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:04.565910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd0c24-494f-47fd-8f4f-a677fbfaa542 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:04.566341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd0c24-494f-47fd-8f4f-a677fbfaa542 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:04.566492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd0c24-494f-47fd-8f4f-a677fbfaa542 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:04.570014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 344/704] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:04.577589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67681a-85bb-403f-81ec-b2308ecb99f7 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:04.578182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67681a-85bb-403f-81ec-b2308ecb99f7 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:04.578646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67681a-85bb-403f-81ec-b2308ecb99f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:04.587974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e8264-76d0-4f8d-9569-fe7f7d954897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:04.589059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e8264-76d0-4f8d-9569-fe7f7d954897 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:04.589538 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e8264-76d0-4f8d-9569-fe7f7d954897 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:04.590035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e8264-76d0-4f8d-9569-fe7f7d954897 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:04.624921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e8264-76d0-4f8d-9569-fe7f7d954897 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:04.628330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e8264-76d0-4f8d-9569-fe7f7d954897 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:04.638308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-7c67681a-85bb-403f-81ec-b2308ecb99f7 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:04.640302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7c67681a-85bb-403f-81ec-b2308ecb99f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-e64e8264-76d0-4f8d-9569-fe7f7d954897 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:04.650089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:04.652835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:04.656474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:04.656474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:04.656474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:04.656474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:04.656474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:04.656474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:04.656474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:04.656474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:04.656474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:07:04.656474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:04.663616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 361/705] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Sep 23 01:07:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:04.663616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e8264-76d0-4f8d-9569-fe7f7d954897 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:04.885512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 345/706] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:07:04.955641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e8264-76d0-4f8d-9569-fe7f7d954897 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:05.015986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67681a-85bb-403f-81ec-b2308ecb99f7 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:05.026305 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef76ff3-b7bf-4302-8c45-364eca6648a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:05.027359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef76ff3-b7bf-4302-8c45-364eca6648a9 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:05.027873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef76ff3-b7bf-4302-8c45-364eca6648a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:05.028314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef76ff3-b7bf-4302-8c45-364eca6648a9 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:05.031262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef76ff3-b7bf-4302-8c45-364eca6648a9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:05.032992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef76ff3-b7bf-4302-8c45-364eca6648a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:05.035534 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef76ff3-b7bf-4302-8c45-364eca6648a9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:05.075598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 362/707] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Sep 23 01:07:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:05.082066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef76ff3-b7bf-4302-8c45-364eca6648a9 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:05.130023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:05.216906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:05.264255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef76ff3-b7bf-4302-8c45-364eca6648a9 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:05.269186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67681a-85bb-403f-81ec-b2308ecb99f7 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:05.321647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c660f4-d644-42db-b7a9-772e23cea97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:05.322640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c660f4-d644-42db-b7a9-772e23cea97e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:05.323091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c660f4-d644-42db-b7a9-772e23cea97e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:05.323543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c660f4-d644-42db-b7a9-772e23cea97e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:05.328293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b8b28-951d-449e-bb5e-b80a9c60a3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:05.329818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b8b28-951d-449e-bb5e-b80a9c60a3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:05.330287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b8b28-951d-449e-bb5e-b80a9c60a3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:05.330743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b8b28-951d-449e-bb5e-b80a9c60a3d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:05.343714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c660f4-d644-42db-b7a9-772e23cea97e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:05.346899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c660f4-d644-42db-b7a9-772e23cea97e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-e5c660f4-d644-42db-b7a9-772e23cea97e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:05.373076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:05.374864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:05.376662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:05.376662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:05.376662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:05.376662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 346/708] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:05.376662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25320f50-f7ba-42b9-8747-b710091c7328 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:05.377207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25320f50-f7ba-42b9-8747-b710091c7328 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:05.377331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25320f50-f7ba-42b9-8747-b710091c7328 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:05.385705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 347/709] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:05.385705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25320f50-f7ba-42b9-8747-b710091c7328 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:05.442366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b8b28-951d-449e-bb5e-b80a9c60a3d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:05.444146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b8b28-951d-449e-bb5e-b80a9c60a3d2 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:05.462465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:05.570508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 363/710] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 23 01:07:05.570508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b8b28-951d-449e-bb5e-b80a9c60a3d2 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:05.582624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25320f50-f7ba-42b9-8747-b710091c7328 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:05.621606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:05.648882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7225284c-5fdf-4ad3-85ab-76e6e0fc0d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:05.652410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225284c-5fdf-4ad3-85ab-76e6e0fc0d7b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:05.652839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225284c-5fdf-4ad3-85ab-76e6e0fc0d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:05.653248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7225284c-5fdf-4ad3-85ab-76e6e0fc0d7b None admin] PATH_INFO: `/v3/services/network` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:05.654125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225284c-5fdf-4ad3-85ab-76e6e0fc0d7b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:05.655742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7225284c-5fdf-4ad3-85ab-76e6e0fc0d7b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-7225284c-5fdf-4ad3-85ab-76e6e0fc0d7b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:05.673883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:05.674969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:05.675961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 348/711] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:05 2022] GET /identity/v3/services/network => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:05.676952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e95442-e25c-4000-a61c-ad0d1cdac8b2 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:05.677569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e95442-e25c-4000-a61c-ad0d1cdac8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:05.677569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e95442-e25c-4000-a61c-ad0d1cdac8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:05.708227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b8b28-951d-449e-bb5e-b80a9c60a3d2 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:05.709136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a0e95442-e25c-4000-a61c-ad0d1cdac8b2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:05.711747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a0e95442-e25c-4000-a61c-ad0d1cdac8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:05.727639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:05.755116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369fd75-bc58-4f6b-b586-8ba3ad06a38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:05.756921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369fd75-bc58-4f6b-b586-8ba3ad06a38b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:05.757416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369fd75-bc58-4f6b-b586-8ba3ad06a38b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:05.758051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369fd75-bc58-4f6b-b586-8ba3ad06a38b None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:05.759302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369fd75-bc58-4f6b-b586-8ba3ad06a38b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:05.759968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369fd75-bc58-4f6b-b586-8ba3ad06a38b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:05.762946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369fd75-bc58-4f6b-b586-8ba3ad06a38b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:05.792001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 364/712] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:05.793233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369fd75-bc58-4f6b-b586-8ba3ad06a38b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:05.840548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 349/713] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:07:05.842745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e95442-e25c-4000-a61c-ad0d1cdac8b2 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:05.989339 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369fd75-bc58-4f6b-b586-8ba3ad06a38b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:06.046035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e95442-e25c-4000-a61c-ad0d1cdac8b2 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:06.059180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc27dd8-cf13-48d1-9be6-f52f5037c961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:06.060356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc27dd8-cf13-48d1-9be6-f52f5037c961 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:06.060861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc27dd8-cf13-48d1-9be6-f52f5037c961 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:06.061354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc27dd8-cf13-48d1-9be6-f52f5037c961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:06.070658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:06.088181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc27dd8-cf13-48d1-9be6-f52f5037c961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:06.088181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc27dd8-cf13-48d1-9be6-f52f5037c961 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:06.099903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b72fae-71d5-456f-805b-e809f458e3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:06.101035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b72fae-71d5-456f-805b-e809f458e3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:06.101540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b72fae-71d5-456f-805b-e809f458e3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:06.102036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b72fae-71d5-456f-805b-e809f458e3e7 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:06.103204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b72fae-71d5-456f-805b-e809f458e3e7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:06.103795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b72fae-71d5-456f-805b-e809f458e3e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:06.106524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b72fae-71d5-456f-805b-e809f458e3e7 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:06.116331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 350/714] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-1bc27dd8-cf13-48d1-9be6-f52f5037c961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:06.117248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:06.118195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:06.119330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:06.119330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:06.119330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:06.121867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 365/715] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:06.123312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b72fae-71d5-456f-805b-e809f458e3e7 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:06.126987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc27dd8-cf13-48d1-9be6-f52f5037c961 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:06.164034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:06.276255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc27dd8-cf13-48d1-9be6-f52f5037c961 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:06.294919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b72fae-71d5-456f-805b-e809f458e3e7 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:06.328825 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb8cfd-7cd1-4174-a276-bc470403d2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:06.329617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb8cfd-7cd1-4174-a276-bc470403d2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:06.329957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb8cfd-7cd1-4174-a276-bc470403d2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:06.330299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb8cfd-7cd1-4174-a276-bc470403d2e6 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:06.331027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb8cfd-7cd1-4174-a276-bc470403d2e6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:06.331342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb8cfd-7cd1-4174-a276-bc470403d2e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:06.332831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb8cfd-7cd1-4174-a276-bc470403d2e6 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:06.340373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 366/716] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:06 2022] GET /identity/v3/services?type=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:06.340775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b700ba-73b6-4406-b42f-c27733e1620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:06.341393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b700ba-73b6-4406-b42f-c27733e1620b None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:06.341678 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b700ba-73b6-4406-b42f-c27733e1620b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:06.341958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b700ba-73b6-4406-b42f-c27733e1620b None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:06.342644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b700ba-73b6-4406-b42f-c27733e1620b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:06.346077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b700ba-73b6-4406-b42f-c27733e1620b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:06.348323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb8cfd-7cd1-4174-a276-bc470403d2e6 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:06.413340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 351/717] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:07:06 2022] POST /identity/v3/users => generated 254 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:07:06.414425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b700ba-73b6-4406-b42f-c27733e1620b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:06.504290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:06.578025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:06.613590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb8cfd-7cd1-4174-a276-bc470403d2e6 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:06.666098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b700ba-73b6-4406-b42f-c27733e1620b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:06.668041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc89a31-3406-4280-8fc1-948e4ee00dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:06.668841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc89a31-3406-4280-8fc1-948e4ee00dbd None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:06.669197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc89a31-3406-4280-8fc1-948e4ee00dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:06.669831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc89a31-3406-4280-8fc1-948e4ee00dbd None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:06.670807 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc89a31-3406-4280-8fc1-948e4ee00dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:06.672359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc89a31-3406-4280-8fc1-948e4ee00dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:06.673955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc89a31-3406-4280-8fc1-948e4ee00dbd None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:06.691681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 367/718] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:06.699127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc89a31-3406-4280-8fc1-948e4ee00dbd None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:06.763841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce79c0-0784-448a-adf4-aba8b335c062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:06.764960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce79c0-0784-448a-adf4-aba8b335c062 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:06.765447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce79c0-0784-448a-adf4-aba8b335c062 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:06.767601 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce79c0-0784-448a-adf4-aba8b335c062 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:06.768526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce79c0-0784-448a-adf4-aba8b335c062 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:06.770974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce79c0-0784-448a-adf4-aba8b335c062 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:06.875745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc89a31-3406-4280-8fc1-948e4ee00dbd None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:06.946000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 352/719] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:07:06 2022] POST /identity/v3/services => generated 251 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:07:06.947413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c6324-8390-4cf1-bbb0-2d785a38c3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:06.948977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c6324-8390-4cf1-bbb0-2d785a38c3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:06.949811 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c6324-8390-4cf1-bbb0-2d785a38c3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:06.950620 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c6324-8390-4cf1-bbb0-2d785a38c3c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:06.968935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c6324-8390-4cf1-bbb0-2d785a38c3c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:06.972957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c6324-8390-4cf1-bbb0-2d785a38c3c8 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-728c6324-8390-4cf1-bbb0-2d785a38c3c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:07.003851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:07.005080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:07.005985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:07.005985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:07.005985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:07.005985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:07.005985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:07.005985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:07.005985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:07.005985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:07.009308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 368/720] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:07.015490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce79c0-0784-448a-adf4-aba8b335c062 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:07.160773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:07.182231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce79c0-0784-448a-adf4-aba8b335c062 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:07.245051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca31d9a-7778-46c3-aefb-b76a15cc96e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:07.246355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca31d9a-7778-46c3-aefb-b76a15cc96e2 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:07.246998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca31d9a-7778-46c3-aefb-b76a15cc96e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:07.247489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca31d9a-7778-46c3-aefb-b76a15cc96e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:07.248639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca31d9a-7778-46c3-aefb-b76a15cc96e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:07.249470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca31d9a-7778-46c3-aefb-b76a15cc96e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:07.252394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca31d9a-7778-46c3-aefb-b76a15cc96e2 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:07.275370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 353/721] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:07.282741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c6324-8390-4cf1-bbb0-2d785a38c3c8 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:07.312236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:07.414846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8eeb4-4713-4217-b692-c2ae8dbb9e3a None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:07.415348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8eeb4-4713-4217-b692-c2ae8dbb9e3a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:07.415626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8eeb4-4713-4217-b692-c2ae8dbb9e3a None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:07.416312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c6324-8390-4cf1-bbb0-2d785a38c3c8 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:07.418636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 354/722] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:07.427791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dfc77-c29b-44f4-a91d-b014626d2eef None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:07.428177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dfc77-c29b-44f4-a91d-b014626d2eef None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:07.428600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dfc77-c29b-44f4-a91d-b014626d2eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:07.454768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-0f0dfc77-c29b-44f4-a91d-b014626d2eef None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:07.456253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0f0dfc77-c29b-44f4-a91d-b014626d2eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:07.474938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5503a9-dfa2-4910-a98d-9316d5957034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:07.476729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5503a9-dfa2-4910-a98d-9316d5957034 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:07.477259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5503a9-dfa2-4910-a98d-9316d5957034 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:07.477776 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5503a9-dfa2-4910-a98d-9316d5957034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:07.479197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5503a9-dfa2-4910-a98d-9316d5957034 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=1c4da5b2130d4fa990adb72f0f6ec2fe) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:07.479646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5503a9-dfa2-4910-a98d-9316d5957034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:07.481441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5503a9-dfa2-4910-a98d-9316d5957034 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:07.491659 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 369/723] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:07:07 2022] GET /identity/v3/role_assignments?user.id=1c4da5b2130d4fa990adb72f0f6ec2fe&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:07.593365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 355/724] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:07:08.028901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5503a9-dfa2-4910-a98d-9316d5957034 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:08.145162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:08.191521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5503a9-dfa2-4910-a98d-9316d5957034 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:08.250334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139724-78ba-4949-ba52-5f2f0e0a677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:08.251878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139724-78ba-4949-ba52-5f2f0e0a677b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:08.252398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139724-78ba-4949-ba52-5f2f0e0a677b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:08.252969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139724-78ba-4949-ba52-5f2f0e0a677b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:08.272313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7139724-78ba-4949-ba52-5f2f0e0a677b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-b7139724-78ba-4949-ba52-5f2f0e0a677b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:08.278369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:08.279477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:08.280380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:08.283244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 370/725] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:07:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:08.289630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dfc77-c29b-44f4-a91d-b014626d2eef None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:08.464285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:08.508479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dfc77-c29b-44f4-a91d-b014626d2eef None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:08.571003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4415c-1d3e-4e75-a74d-98599495fd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:08.572593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4415c-1d3e-4e75-a74d-98599495fd50 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:08.573599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4415c-1d3e-4e75-a74d-98599495fd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:08.574343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4415c-1d3e-4e75-a74d-98599495fd50 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:08.575823 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4415c-1d3e-4e75-a74d-98599495fd50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:08.576671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4415c-1d3e-4e75-a74d-98599495fd50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:08.579319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4415c-1d3e-4e75-a74d-98599495fd50 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:08.596372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 356/726] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:07:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:08.606232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139724-78ba-4949-ba52-5f2f0e0a677b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:08.777845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139724-78ba-4949-ba52-5f2f0e0a677b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:08.826410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:08.845356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22379fd-0471-4d13-97da-b86a65b786a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:08.846313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22379fd-0471-4d13-97da-b86a65b786a1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:08.846783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22379fd-0471-4d13-97da-b86a65b786a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:08.847346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22379fd-0471-4d13-97da-b86a65b786a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:08.866560 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22379fd-0471-4d13-97da-b86a65b786a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:08.869151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22379fd-0471-4d13-97da-b86a65b786a1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-e22379fd-0471-4d13-97da-b86a65b786a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:08.893923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:08.895059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:08.896195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:08.896195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:08.896195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:08.898008 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 371/727] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:08.908330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4415c-1d3e-4e75-a74d-98599495fd50 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:09.149560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4415c-1d3e-4e75-a74d-98599495fd50 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:09.210404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8083b2-e7bf-422e-aec5-da7137022c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:09.211056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8083b2-e7bf-422e-aec5-da7137022c39 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:09.211507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8083b2-e7bf-422e-aec5-da7137022c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:09.211921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8083b2-e7bf-422e-aec5-da7137022c39 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:09.212670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8083b2-e7bf-422e-aec5-da7137022c39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:09.212976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8083b2-e7bf-422e-aec5-da7137022c39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:09.214492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8083b2-e7bf-422e-aec5-da7137022c39 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:09.231849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 357/728] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:09.240615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22379fd-0471-4d13-97da-b86a65b786a1 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:09.424679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd41d1-00ed-4747-86ac-b598f4e26cef None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:09.424874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd41d1-00ed-4747-86ac-b598f4e26cef None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:09.425141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd41d1-00ed-4747-86ac-b598f4e26cef None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:09.426946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 358/729] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:09.433878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd14488-06d6-474b-902a-c2730dd78d88 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:09.434369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd14488-06d6-474b-902a-c2730dd78d88 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:09.434844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd14488-06d6-474b-902a-c2730dd78d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:09.445463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22379fd-0471-4d13-97da-b86a65b786a1 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:09.463954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:09.487121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-dbd14488-06d6-474b-902a-c2730dd78d88 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:09.489768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-dbd14488-06d6-474b-902a-c2730dd78d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:09.538278 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d504043-0b11-499a-804d-677bc5a02268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:09.539025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d504043-0b11-499a-804d-677bc5a02268 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:09.539308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d504043-0b11-499a-804d-677bc5a02268 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:09.539610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d504043-0b11-499a-804d-677bc5a02268 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:09.555585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d504043-0b11-499a-804d-677bc5a02268 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:09.557134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d504043-0b11-499a-804d-677bc5a02268 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-7d504043-0b11-499a-804d-677bc5a02268 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:09.581663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:09.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:09.583904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 372/730] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:07:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:09.592262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d504043-0b11-499a-804d-677bc5a02268 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:09.650147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 359/731] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 23 01:07:09.695748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:09.727761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9f4bc-ec2e-4445-93fb-f30489620046 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:09.728444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9f4bc-ec2e-4445-93fb-f30489620046 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:09.728927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9f4bc-ec2e-4445-93fb-f30489620046 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:09.730989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 360/732] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:09.738210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cb1eb-91a8-4144-80dd-ed48f7fd6512 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:09.738896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cb1eb-91a8-4144-80dd-ed48f7fd6512 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:09.739666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cb1eb-91a8-4144-80dd-ed48f7fd6512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:09.777332 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-7f0cb1eb-91a8-4144-80dd-ed48f7fd6512 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:09.778891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7f0cb1eb-91a8-4144-80dd-ed48f7fd6512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:09.806873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d504043-0b11-499a-804d-677bc5a02268 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:09.875958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7dc8b7-0735-4ef9-a60d-6d9f3f70989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:09.875958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7dc8b7-0735-4ef9-a60d-6d9f3f70989f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:09.875958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7dc8b7-0735-4ef9-a60d-6d9f3f70989f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:09.875958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7dc8b7-0735-4ef9-a60d-6d9f3f70989f None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:09.876462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7dc8b7-0735-4ef9-a60d-6d9f3f70989f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:09.876462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7dc8b7-0735-4ef9-a60d-6d9f3f70989f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:09.876462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7dc8b7-0735-4ef9-a60d-6d9f3f70989f None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:09.911142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 373/733] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Sep 23 01:07:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:09.920660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7dc8b7-0735-4ef9-a60d-6d9f3f70989f None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:09.969516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 361/734] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:09 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 23 01:07:10.119453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7dc8b7-0735-4ef9-a60d-6d9f3f70989f None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:10.122449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:10.188252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20f37d-99b3-4270-9778-dcfd16b2bc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:10.189404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20f37d-99b3-4270-9778-dcfd16b2bc73 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:10.189953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20f37d-99b3-4270-9778-dcfd16b2bc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:10.190440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20f37d-99b3-4270-9778-dcfd16b2bc73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:10.229726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cb1eb-91a8-4144-80dd-ed48f7fd6512 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:10.332119 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20f37d-99b3-4270-9778-dcfd16b2bc73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:10.334940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20f37d-99b3-4270-9778-dcfd16b2bc73 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:10.403921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:10.451582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cb1eb-91a8-4144-80dd-ed48f7fd6512 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:10.503405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ab41e-d7e1-478d-af6a-b948ef4fc0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:10.504050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ab41e-d7e1-478d-af6a-b948ef4fc0ef None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:10.504449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ab41e-d7e1-478d-af6a-b948ef4fc0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:10.505016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8ab41e-d7e1-478d-af6a-b948ef4fc0ef None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:10.505874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ab41e-d7e1-478d-af6a-b948ef4fc0ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:10.507435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8ab41e-d7e1-478d-af6a-b948ef4fc0ef None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:10.550781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 374/735] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:07:09 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 23 01:07:10.552419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20f37d-99b3-4270-9778-dcfd16b2bc73 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:10.557059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:10.603926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 362/736] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:07:10 2022] POST /identity/v3/services => generated 240 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:07:10.604790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ab41e-d7e1-478d-af6a-b948ef4fc0ef None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:10.728513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20f37d-99b3-4270-9778-dcfd16b2bc73 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:10.811880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95cae68-5a17-42c6-99cb-0f8cfa83b66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:10.813281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cae68-5a17-42c6-99cb-0f8cfa83b66e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:10.813753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cae68-5a17-42c6-99cb-0f8cfa83b66e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:10.814223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95cae68-5a17-42c6-99cb-0f8cfa83b66e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:10.828861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8ab41e-d7e1-478d-af6a-b948ef4fc0ef None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:10.836390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95cae68-5a17-42c6-99cb-0f8cfa83b66e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-d95cae68-5a17-42c6-99cb-0f8cfa83b66e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:10.842486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:10.843789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:10.844856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 375/737] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:07:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:10.845671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2123120-a1f6-4897-aefd-9635621217d4 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:10.845671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2123120-a1f6-4897-aefd-9635621217d4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:10.845806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2123120-a1f6-4897-aefd-9635621217d4 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:10.847049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 376/738] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:10.853972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05923083-a4d5-49c4-9b8f-95c9bb17634c None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:10.853972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05923083-a4d5-49c4-9b8f-95c9bb17634c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:10.853972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05923083-a4d5-49c4-9b8f-95c9bb17634c None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:10.853972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 377/739] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:10.853972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05923083-a4d5-49c4-9b8f-95c9bb17634c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:10.880802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:10.897752 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fea6f4-fe55-4728-8523-755afbb1d6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:10.898647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea6f4-fe55-4728-8523-755afbb1d6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:10.898909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea6f4-fe55-4728-8523-755afbb1d6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:10.899186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fea6f4-fe55-4728-8523-755afbb1d6d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:10.914643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea6f4-fe55-4728-8523-755afbb1d6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:10.916223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fea6f4-fe55-4728-8523-755afbb1d6d0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-e5fea6f4-fe55-4728-8523-755afbb1d6d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:10.933910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:10.936272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:10.939244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:10.939244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:10.939244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:10.939244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 363/740] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:10.939244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44755f-0fac-42fa-bd5c-77c4144e3e79 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:10.939244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44755f-0fac-42fa-bd5c-77c4144e3e79 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:10.939244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44755f-0fac-42fa-bd5c-77c4144e3e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:10.956891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:10.970115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-de44755f-0fac-42fa-bd5c-77c4144e3e79 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:10.970742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-de44755f-0fac-42fa-bd5c-77c4144e3e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:11.017602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05923083-a4d5-49c4-9b8f-95c9bb17634c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:11.074339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 364/741] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:11.074339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91fa2f-85cc-4ecf-8fa2-0f4bebefc664 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:11.074339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91fa2f-85cc-4ecf-8fa2-0f4bebefc664 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:11.074339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c91fa2f-85cc-4ecf-8fa2-0f4bebefc664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:11.078083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85ad5d-6033-4ac3-888c-b5f5a24abcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:11.079900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ad5d-6033-4ac3-888c-b5f5a24abcae None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:11.080367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ad5d-6033-4ac3-888c-b5f5a24abcae None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:11.080977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85ad5d-6033-4ac3-888c-b5f5a24abcae None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:11.082170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85ad5d-6033-4ac3-888c-b5f5a24abcae None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:11.082783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85ad5d-6033-4ac3-888c-b5f5a24abcae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:11.085524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85ad5d-6033-4ac3-888c-b5f5a24abcae None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:11.096300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-1c91fa2f-85cc-4ecf-8fa2-0f4bebefc664 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:11.097996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1c91fa2f-85cc-4ecf-8fa2-0f4bebefc664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:11.103537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 378/742] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:07:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:11.105119 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85ad5d-6033-4ac3-888c-b5f5a24abcae None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:11.226389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 365/743] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:11.228646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91fa2f-85cc-4ecf-8fa2-0f4bebefc664 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:11.311010 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85ad5d-6033-4ac3-888c-b5f5a24abcae None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:11.319460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:11.364115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:11.386268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d79b7-a142-4ba2-940d-4b84b77702cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:11.386907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d79b7-a142-4ba2-940d-4b84b77702cc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:11.387242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d79b7-a142-4ba2-940d-4b84b77702cc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:11.387523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d79b7-a142-4ba2-940d-4b84b77702cc None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:11.388335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d79b7-a142-4ba2-940d-4b84b77702cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:11.388755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d79b7-a142-4ba2-940d-4b84b77702cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:11.393769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d79b7-a142-4ba2-940d-4b84b77702cc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:11.420050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 379/744] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:11.421905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d79b7-a142-4ba2-940d-4b84b77702cc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:11.460521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c91fa2f-85cc-4ecf-8fa2-0f4bebefc664 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:11.521003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7861e9d-0de3-428b-913e-a1c658e1af19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:11.521684 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7861e9d-0de3-428b-913e-a1c658e1af19 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:11.521938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7861e9d-0de3-428b-913e-a1c658e1af19 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:11.522201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7861e9d-0de3-428b-913e-a1c658e1af19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:11.539390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7861e9d-0de3-428b-913e-a1c658e1af19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:11.541199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7861e9d-0de3-428b-913e-a1c658e1af19 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-a7861e9d-0de3-428b-913e-a1c658e1af19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:11.559592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:11.560812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:11.561794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:11.561794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:11.561794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:11.563442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 366/745] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:11.564339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7861e9d-0de3-428b-913e-a1c658e1af19 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:11.589471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d79b7-a142-4ba2-940d-4b84b77702cc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:11.641365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e97f72-0fc8-4e07-bc9e-b4243847a960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:11.642522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e97f72-0fc8-4e07-bc9e-b4243847a960 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:11.643073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e97f72-0fc8-4e07-bc9e-b4243847a960 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:11.643939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e97f72-0fc8-4e07-bc9e-b4243847a960 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:11.644865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e97f72-0fc8-4e07-bc9e-b4243847a960 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:11.647579 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e97f72-0fc8-4e07-bc9e-b4243847a960 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-24e97f72-0fc8-4e07-bc9e-b4243847a960 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:11.659913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:11.661106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:11.662159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 380/746] 10.222.0.64 () {56 vars in 1129 bytes} [Fri Sep 23 01:07:11 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:11.663151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e97f72-0fc8-4e07-bc9e-b4243847a960 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:11.696399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:11.726616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7861e9d-0de3-428b-913e-a1c658e1af19 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:11.737094 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:11.795936 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25987-ce26-45c0-aec0-c2c52c2235af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:11.797839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25987-ce26-45c0-aec0-c2c52c2235af None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:11.798379 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25987-ce26-45c0-aec0-c2c52c2235af None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:11.798749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25987-ce26-45c0-aec0-c2c52c2235af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:11.810420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b25987-ce26-45c0-aec0-c2c52c2235af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:11.812941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b25987-ce26-45c0-aec0-c2c52c2235af None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-e0b25987-ce26-45c0-aec0-c2c52c2235af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:11.841574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:11.843336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:11.846187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:11.846187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:11.846187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:11.846187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:11.846187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:11.846187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:11.846187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:11.846187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:11.847093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e97f72-0fc8-4e07-bc9e-b4243847a960 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:11.848177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 367/747] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:11.850746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25987-ce26-45c0-aec0-c2c52c2235af None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:11.893440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbaf97-f5df-4924-8b5e-c22eb478880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:11.894066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbaf97-f5df-4924-8b5e-c22eb478880e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:11.894479 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbaf97-f5df-4924-8b5e-c22eb478880e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:11.894869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbaf97-f5df-4924-8b5e-c22eb478880e None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:11.895970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbaf97-f5df-4924-8b5e-c22eb478880e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:11.896481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbaf97-f5df-4924-8b5e-c22eb478880e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:11.898483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbaf97-f5df-4924-8b5e-c22eb478880e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:11.926940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 381/748] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:11.926940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbaf97-f5df-4924-8b5e-c22eb478880e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:12.028003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25987-ce26-45c0-aec0-c2c52c2235af None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:12.082231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054c71a-e820-436e-bd52-307f68c67c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:12.083279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054c71a-e820-436e-bd52-307f68c67c82 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:12.083560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054c71a-e820-436e-bd52-307f68c67c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:12.083849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054c71a-e820-436e-bd52-307f68c67c82 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:12.084665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054c71a-e820-436e-bd52-307f68c67c82 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:12.085056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054c71a-e820-436e-bd52-307f68c67c82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:12.086454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054c71a-e820-436e-bd52-307f68c67c82 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:12.092514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 368/749] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Sep 23 01:07:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:12.093663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054c71a-e820-436e-bd52-307f68c67c82 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:12.097393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbaf97-f5df-4924-8b5e-c22eb478880e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:12.105910 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:12.114609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:12.151687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f1800-c992-49da-b198-c7a66f2032ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:12.152317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f1800-c992-49da-b198-c7a66f2032ac None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:12.152606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f1800-c992-49da-b198-c7a66f2032ac None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:12.152880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f1800-c992-49da-b198-c7a66f2032ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:12.164546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f1800-c992-49da-b198-c7a66f2032ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-460f1800-c992-49da-b198-c7a66f2032ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:12.169566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:12.170710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:12.172524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:12.175730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 382/750] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:07:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:12.177102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f1800-c992-49da-b198-c7a66f2032ac None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:12.257947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054c71a-e820-436e-bd52-307f68c67c82 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:12.300692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4095e-3137-4fd8-b898-de831f359ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:12.301595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4095e-3137-4fd8-b898-de831f359ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:12.302050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4095e-3137-4fd8-b898-de831f359ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:12.302138 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4095e-3137-4fd8-b898-de831f359ae0 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:12.302937 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4095e-3137-4fd8-b898-de831f359ae0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:12.303727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4095e-3137-4fd8-b898-de831f359ae0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:12.305511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4095e-3137-4fd8-b898-de831f359ae0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:12.320923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 369/751] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:12.321644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4095e-3137-4fd8-b898-de831f359ae0 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:12.330784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f1800-c992-49da-b198-c7a66f2032ac None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:12.377803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b16f9-66c0-4d75-94c0-5aeaec9a68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:12.379292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b16f9-66c0-4d75-94c0-5aeaec9a68dd None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:12.379850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b16f9-66c0-4d75-94c0-5aeaec9a68dd None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:12.380132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b16f9-66c0-4d75-94c0-5aeaec9a68dd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:12.394182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b16f9-66c0-4d75-94c0-5aeaec9a68dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:12.395844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b16f9-66c0-4d75-94c0-5aeaec9a68dd None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-3a1b16f9-66c0-4d75-94c0-5aeaec9a68dd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:12.418809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:12.420719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:12.425013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 383/752] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Sep 23 01:07:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:12.426857 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:12.427136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b16f9-66c0-4d75-94c0-5aeaec9a68dd None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:12.476621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:12.481154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4095e-3137-4fd8-b898-de831f359ae0 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:12.527344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58b413-c309-4b15-b9f9-ada009419a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:12.527344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58b413-c309-4b15-b9f9-ada009419a64 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:12.528473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58b413-c309-4b15-b9f9-ada009419a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:12.528473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58b413-c309-4b15-b9f9-ada009419a64 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:12.528909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58b413-c309-4b15-b9f9-ada009419a64 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:12.529279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58b413-c309-4b15-b9f9-ada009419a64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:12.531152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58b413-c309-4b15-b9f9-ada009419a64 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:12.540745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 370/753] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Sep 23 01:07:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:12.542129 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58b413-c309-4b15-b9f9-ada009419a64 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:12.620497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b16f9-66c0-4d75-94c0-5aeaec9a68dd None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:12.670202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4967b8f-3741-4151-b521-492ca4fa6de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:12.670792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4967b8f-3741-4151-b521-492ca4fa6de5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:12.671114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4967b8f-3741-4151-b521-492ca4fa6de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:12.671400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4967b8f-3741-4151-b521-492ca4fa6de5 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:12.672683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4967b8f-3741-4151-b521-492ca4fa6de5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:12.673006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4967b8f-3741-4151-b521-492ca4fa6de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:12.674430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4967b8f-3741-4151-b521-492ca4fa6de5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:12.684492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 384/754] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:07:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:12.689797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4967b8f-3741-4151-b521-492ca4fa6de5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:12.727982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58b413-c309-4b15-b9f9-ada009419a64 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:12.791504 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482879ea-d80f-490a-a6c6-19aadb0f6950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:12.792603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482879ea-d80f-490a-a6c6-19aadb0f6950 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:12.793073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482879ea-d80f-490a-a6c6-19aadb0f6950 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:12.793642 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482879ea-d80f-490a-a6c6-19aadb0f6950 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:12.795018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482879ea-d80f-490a-a6c6-19aadb0f6950 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=c5a9f3eca3e34fffa7b218064e831d7b) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:12.795590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482879ea-d80f-490a-a6c6-19aadb0f6950 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:12.798739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482879ea-d80f-490a-a6c6-19aadb0f6950 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:12.802205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:12.819580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 371/755] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:07:12 2022] GET /identity/v3/role_assignments?user.id=c5a9f3eca3e34fffa7b218064e831d7b&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:12.819580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482879ea-d80f-490a-a6c6-19aadb0f6950 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:12.860641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:12.892799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4967b8f-3741-4151-b521-492ca4fa6de5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:12.975078 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d83ee-6a50-4b23-ab99-2b9139f20d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:12.976108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d83ee-6a50-4b23-ab99-2b9139f20d0c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:12.976554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d83ee-6a50-4b23-ab99-2b9139f20d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:12.977010 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d83ee-6a50-4b23-ab99-2b9139f20d0c None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:12.978478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d83ee-6a50-4b23-ab99-2b9139f20d0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:12.979023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d83ee-6a50-4b23-ab99-2b9139f20d0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:12.981542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d83ee-6a50-4b23-ab99-2b9139f20d0c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:13.017778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 385/756] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Sep 23 01:07:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:13.019577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d83ee-6a50-4b23-ab99-2b9139f20d0c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:13.051345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482879ea-d80f-490a-a6c6-19aadb0f6950 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:13.118806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd5b90-4eed-46e4-9627-97a957b6f264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:13.120513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd5b90-4eed-46e4-9627-97a957b6f264 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:13.121388 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd5b90-4eed-46e4-9627-97a957b6f264 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:13.122194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd5b90-4eed-46e4-9627-97a957b6f264 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:13.123979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd5b90-4eed-46e4-9627-97a957b6f264 None admin] RBAC: Adding query filter params (interface=public, service_id=694cb74cc3de4903b6322d009e0efe53, region_id=RegionOne) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:13.124921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd5b90-4eed-46e4-9627-97a957b6f264 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:13.127986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd5b90-4eed-46e4-9627-97a957b6f264 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:13.151463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 372/757] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Sep 23 01:07:12 2022] GET /identity/v3/endpoints?service_id=694cb74cc3de4903b6322d009e0efe53&interface=public®ion_id=RegionOne => generated 194 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:13.153600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd5b90-4eed-46e4-9627-97a957b6f264 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:13.179242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d83ee-6a50-4b23-ab99-2b9139f20d0c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:13.222255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c336bd4-4b1e-47ce-a48e-624b4d8f4bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:13.222255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c336bd4-4b1e-47ce-a48e-624b4d8f4bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:13.222684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c336bd4-4b1e-47ce-a48e-624b4d8f4bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:13.222814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c336bd4-4b1e-47ce-a48e-624b4d8f4bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:13.236691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c336bd4-4b1e-47ce-a48e-624b4d8f4bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:13.238178 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c336bd4-4b1e-47ce-a48e-624b4d8f4bf9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:13.241926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-1c336bd4-4b1e-47ce-a48e-624b4d8f4bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:13.258544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:13.259718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:13.260677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:13.260677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:13.260677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:13.260677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 386/758] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:13.261195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c336bd4-4b1e-47ce-a48e-624b4d8f4bf9 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:13.278984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:13.364862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd5b90-4eed-46e4-9627-97a957b6f264 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:13.427427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c336bd4-4b1e-47ce-a48e-624b4d8f4bf9 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:13.455958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1e5c4-3c82-4e7e-ab4e-8e07ddeacd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:13.456852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1e5c4-3c82-4e7e-ab4e-8e07ddeacd48 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:13.457192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1e5c4-3c82-4e7e-ab4e-8e07ddeacd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:13.457468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1e5c4-3c82-4e7e-ab4e-8e07ddeacd48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:13.489195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb5a42e-3db3-4a50-a3ab-453ec4f0c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:13.490029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5a42e-3db3-4a50-a3ab-453ec4f0c6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:13.490500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5a42e-3db3-4a50-a3ab-453ec4f0c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:13.490942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb5a42e-3db3-4a50-a3ab-453ec4f0c6c1 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:13.491774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5a42e-3db3-4a50-a3ab-453ec4f0c6c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:13.493500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb5a42e-3db3-4a50-a3ab-453ec4f0c6c1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:13.507583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 387/759] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Sep 23 01:07:13 2022] GET /identity/v3/services => generated 772 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:13.507583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f6719-e00d-4b73-a748-ee492608f223 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:13.507583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f6719-e00d-4b73-a748-ee492608f223 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:13.507583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f6719-e00d-4b73-a748-ee492608f223 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:13.507583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 388/760] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:13.507583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f6719-e00d-4b73-a748-ee492608f223 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:13.611584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1e5c4-3c82-4e7e-ab4e-8e07ddeacd48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:13.614817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1e5c4-3c82-4e7e-ab4e-8e07ddeacd48 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:13.632132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:13.678716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f6719-e00d-4b73-a748-ee492608f223 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:13.737438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f28b2-0b5d-48e4-8065-68e77ba19ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:13.738617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f28b2-0b5d-48e4-8065-68e77ba19ccf None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:13.739117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f28b2-0b5d-48e4-8065-68e77ba19ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:13.739698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f28b2-0b5d-48e4-8065-68e77ba19ccf None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:13.740748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f28b2-0b5d-48e4-8065-68e77ba19ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:13.741296 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f28b2-0b5d-48e4-8065-68e77ba19ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:13.744034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f28b2-0b5d-48e4-8065-68e77ba19ccf None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:13.780194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:13.869607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 373/761] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:07:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:07:13.871769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c99dd-d7d8-4a00-9a7a-7abd20315407 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:13.872206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c99dd-d7d8-4a00-9a7a-7abd20315407 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:13.872642 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c99dd-d7d8-4a00-9a7a-7abd20315407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:13.902232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-c86c99dd-d7d8-4a00-9a7a-7abd20315407 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:13.907628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c86c99dd-d7d8-4a00-9a7a-7abd20315407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:13.957275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 389/762] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:13.958664 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f28b2-0b5d-48e4-8065-68e77ba19ccf None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:14.029288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 374/763] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:14.032242 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c99dd-d7d8-4a00-9a7a-7abd20315407 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:14.197864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f28b2-0b5d-48e4-8065-68e77ba19ccf None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:14.271485 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:14.291346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:14.298001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf4183-3964-42b1-811c-dfcd632f39ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:14.299048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf4183-3964-42b1-811c-dfcd632f39ef None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:14.299498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf4183-3964-42b1-811c-dfcd632f39ef None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:14.299961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf4183-3964-42b1-811c-dfcd632f39ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:14.309662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c99dd-d7d8-4a00-9a7a-7abd20315407 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:14.340828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf4183-3964-42b1-811c-dfcd632f39ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:14.343402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf4183-3964-42b1-811c-dfcd632f39ef None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-93bf4183-3964-42b1-811c-dfcd632f39ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:14.391372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:14.392470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:14.393373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:14.393373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:14.393373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:14.401741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 390/764] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:14.408630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf4183-3964-42b1-811c-dfcd632f39ef None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:14.409963 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7647-1244-4df5-a0dd-7c458279a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:14.411015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7647-1244-4df5-a0dd-7c458279a58f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:14.411508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7647-1244-4df5-a0dd-7c458279a58f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:14.412051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b7647-1244-4df5-a0dd-7c458279a58f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:14.425946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b7647-1244-4df5-a0dd-7c458279a58f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:14.428566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b7647-1244-4df5-a0dd-7c458279a58f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-e22b7647-1244-4df5-a0dd-7c458279a58f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:14.451347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:14.452400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:14.453582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:14.453582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:14.453582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:14.453582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:14.453582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:14.453582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:14.453582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:14.453582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:14.453582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:07:14.453582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:14.459791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 375/765] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Sep 23 01:07:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:14.462787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7647-1244-4df5-a0dd-7c458279a58f None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:14.664366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b7647-1244-4df5-a0dd-7c458279a58f None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:14.693542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf4183-3964-42b1-811c-dfcd632f39ef None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:14.711409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74ee01-e66f-444e-a594-6b00baf235e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:14.712267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74ee01-e66f-444e-a594-6b00baf235e1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:14.712556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74ee01-e66f-444e-a594-6b00baf235e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:14.712838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74ee01-e66f-444e-a594-6b00baf235e1 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:14.713671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74ee01-e66f-444e-a594-6b00baf235e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:14.713989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74ee01-e66f-444e-a594-6b00baf235e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:14.715729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74ee01-e66f-444e-a594-6b00baf235e1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:14.747580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 376/766] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Sep 23 01:07:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:14.747580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74ee01-e66f-444e-a594-6b00baf235e1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:14.750936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adf46f-e7a0-4a0b-9168-0e4ecc460002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:14.752069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adf46f-e7a0-4a0b-9168-0e4ecc460002 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:14.752652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adf46f-e7a0-4a0b-9168-0e4ecc460002 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:14.753165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adf46f-e7a0-4a0b-9168-0e4ecc460002 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:14.754156 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adf46f-e7a0-4a0b-9168-0e4ecc460002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:14.754721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adf46f-e7a0-4a0b-9168-0e4ecc460002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:14.756657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adf46f-e7a0-4a0b-9168-0e4ecc460002 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:14.764863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:14.782061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 391/767] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:14.783619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adf46f-e7a0-4a0b-9168-0e4ecc460002 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:14.862250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:14.935220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74ee01-e66f-444e-a594-6b00baf235e1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:15.004425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adf46f-e7a0-4a0b-9168-0e4ecc460002 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:15.014701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78878a-e8f8-4d9c-bd2c-c8f67737892a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:15.016187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78878a-e8f8-4d9c-bd2c-c8f67737892a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:15.017496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78878a-e8f8-4d9c-bd2c-c8f67737892a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:15.017962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78878a-e8f8-4d9c-bd2c-c8f67737892a None admin] PATH_INFO: `/v3/services/network` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:15.019014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78878a-e8f8-4d9c-bd2c-c8f67737892a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:15.022380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78878a-e8f8-4d9c-bd2c-c8f67737892a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-9d78878a-e8f8-4d9c-bd2c-c8f67737892a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:15.044614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:15.046956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:15.049150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 377/768] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:14 2022] GET /identity/v3/services/network => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:15.050534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78878a-e8f8-4d9c-bd2c-c8f67737892a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:15.061292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e289f69-264d-40d6-b240-7330fe8cf2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:15.061962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289f69-264d-40d6-b240-7330fe8cf2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:15.062262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289f69-264d-40d6-b240-7330fe8cf2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:15.062552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e289f69-264d-40d6-b240-7330fe8cf2e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:15.079090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e289f69-264d-40d6-b240-7330fe8cf2e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:15.080755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e289f69-264d-40d6-b240-7330fe8cf2e7 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-8e289f69-264d-40d6-b240-7330fe8cf2e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:15.111867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:15.113708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:15.115429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:15.115429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:15.115429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:15.124652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 392/769] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:15.124652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e289f69-264d-40d6-b240-7330fe8cf2e7 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:15.190900 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:15.246623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78878a-e8f8-4d9c-bd2c-c8f67737892a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:15.309370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:15.329662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b269473-e2d7-4cf5-91f0-184aab7b5e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:15.331434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269473-e2d7-4cf5-91f0-184aab7b5e98 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:15.332243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269473-e2d7-4cf5-91f0-184aab7b5e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:15.333598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b269473-e2d7-4cf5-91f0-184aab7b5e98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:15.344082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e289f69-264d-40d6-b240-7330fe8cf2e7 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:15.353713 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b269473-e2d7-4cf5-91f0-184aab7b5e98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:15.356903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b269473-e2d7-4cf5-91f0-184aab7b5e98 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-4b269473-e2d7-4cf5-91f0-184aab7b5e98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:15.384757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:15.386445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:15.388170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:15.388170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:15.388170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:15.388170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:15.388170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:15.388170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:15.388170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:15.388170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:15.391991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 378/770] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:15.393893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b269473-e2d7-4cf5-91f0-184aab7b5e98 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:15.421687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00395c-3fa3-4c25-a072-3e834120938b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:15.423343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00395c-3fa3-4c25-a072-3e834120938b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:15.423870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00395c-3fa3-4c25-a072-3e834120938b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:15.424374 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00395c-3fa3-4c25-a072-3e834120938b None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:15.425185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00395c-3fa3-4c25-a072-3e834120938b None admin] RBAC: Adding query filter params (name=network) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:15.425643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00395c-3fa3-4c25-a072-3e834120938b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:15.428709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00395c-3fa3-4c25-a072-3e834120938b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:15.436545 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 393/771] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:15.438474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00395c-3fa3-4c25-a072-3e834120938b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:15.545052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b269473-e2d7-4cf5-91f0-184aab7b5e98 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:15.595377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a67b2-4e0d-465c-905e-58bd4da46aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:15.596517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a67b2-4e0d-465c-905e-58bd4da46aba None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:15.596684 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a67b2-4e0d-465c-905e-58bd4da46aba None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:15.597300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3a67b2-4e0d-465c-905e-58bd4da46aba None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:15.598053 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a67b2-4e0d-465c-905e-58bd4da46aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:15.598516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a67b2-4e0d-465c-905e-58bd4da46aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:15.600036 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00395c-3fa3-4c25-a072-3e834120938b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:15.600368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3a67b2-4e0d-465c-905e-58bd4da46aba None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:15.602623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:15.618381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 379/772] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:15.618381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a67b2-4e0d-465c-905e-58bd4da46aba None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:15.658567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780cb5c-d4a9-4a02-bee8-abfc9ae242bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:15.659712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780cb5c-d4a9-4a02-bee8-abfc9ae242bc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:15.660250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780cb5c-d4a9-4a02-bee8-abfc9ae242bc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:15.660815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780cb5c-d4a9-4a02-bee8-abfc9ae242bc None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:15.661732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780cb5c-d4a9-4a02-bee8-abfc9ae242bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:15.662455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780cb5c-d4a9-4a02-bee8-abfc9ae242bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:15.664098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780cb5c-d4a9-4a02-bee8-abfc9ae242bc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:15.684965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 394/773] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:15.686217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780cb5c-d4a9-4a02-bee8-abfc9ae242bc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:15.699887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:15.762163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3a67b2-4e0d-465c-905e-58bd4da46aba None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:15.804969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cdf43-335a-418a-83f7-25ee18d7dc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:15.805508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cdf43-335a-418a-83f7-25ee18d7dc90 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:15.805820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cdf43-335a-418a-83f7-25ee18d7dc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:15.806101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6cdf43-335a-418a-83f7-25ee18d7dc90 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:15.806765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cdf43-335a-418a-83f7-25ee18d7dc90 None admin] RBAC: Adding query filter params (type=network) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:15.807234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cdf43-335a-418a-83f7-25ee18d7dc90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:15.811685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6cdf43-335a-418a-83f7-25ee18d7dc90 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:15.818289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780cb5c-d4a9-4a02-bee8-abfc9ae242bc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:15.827615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 380/774] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:15.827615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cdf43-335a-418a-83f7-25ee18d7dc90 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:15.864958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0c2d6-b116-4745-9710-81758ca56f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:15.865652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0c2d6-b116-4745-9710-81758ca56f1d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:15.865928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0c2d6-b116-4745-9710-81758ca56f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:15.866204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0c2d6-b116-4745-9710-81758ca56f1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:15.878978 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0c2d6-b116-4745-9710-81758ca56f1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:15.882827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0c2d6-b116-4745-9710-81758ca56f1d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-a6c0c2d6-b116-4745-9710-81758ca56f1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:15.895778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:15.896905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:15.897891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:15.897891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:15.897891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:15.897891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:15.897891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:15.897891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:15.897891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:15.897891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:15.897891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 395/775] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:15.898844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521500f-70a2-4b0b-aa06-2d2e2c6f82a1 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:15.898844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521500f-70a2-4b0b-aa06-2d2e2c6f82a1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:15.899081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521500f-70a2-4b0b-aa06-2d2e2c6f82a1 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:15.900838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 396/776] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:15.901188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521500f-70a2-4b0b-aa06-2d2e2c6f82a1 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:15.940068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:16.010330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6cdf43-335a-418a-83f7-25ee18d7dc90 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:16.022475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:16.067332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521500f-70a2-4b0b-aa06-2d2e2c6f82a1 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:16.069957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc041712-ef8a-4943-baa7-ab567512e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:16.071007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc041712-ef8a-4943-baa7-ab567512e426 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:16.071509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc041712-ef8a-4943-baa7-ab567512e426 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:16.072041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc041712-ef8a-4943-baa7-ab567512e426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:16.073454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc041712-ef8a-4943-baa7-ab567512e426 None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=fa998fcd51b3404aab5412323f86908f) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:16.074004 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc041712-ef8a-4943-baa7-ab567512e426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:16.076767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc041712-ef8a-4943-baa7-ab567512e426 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:16.087774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 381/777] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:07:15 2022] GET /identity/v3/role_assignments?user.id=fa998fcd51b3404aab5412323f86908f&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:16.089277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc041712-ef8a-4943-baa7-ab567512e426 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:16.136675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0be7f8-6756-4a9b-b6ae-2f5ffb4a9302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:16.137636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0be7f8-6756-4a9b-b6ae-2f5ffb4a9302 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:16.138124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0be7f8-6756-4a9b-b6ae-2f5ffb4a9302 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:16.138616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0be7f8-6756-4a9b-b6ae-2f5ffb4a9302 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:16.140001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0be7f8-6756-4a9b-b6ae-2f5ffb4a9302 None admin] RBAC: Adding query filter params (interface=public, service_id=3f387ef2932c478db4a4b2afe38e1868, region_id=RegionOne) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:16.140562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0be7f8-6756-4a9b-b6ae-2f5ffb4a9302 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:16.143277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0be7f8-6756-4a9b-b6ae-2f5ffb4a9302 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:16.163645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 397/778] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Sep 23 01:07:15 2022] GET /identity/v3/endpoints?service_id=3f387ef2932c478db4a4b2afe38e1868&interface=public®ion_id=RegionOne => generated 194 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:16.163645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdf79-e774-4a03-90e0-efd2eddb11d4 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:16.163645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdf79-e774-4a03-90e0-efd2eddb11d4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:16.163645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bdf79-e774-4a03-90e0-efd2eddb11d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:16.195869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-7f6bdf79-e774-4a03-90e0-efd2eddb11d4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:16.198013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7f6bdf79-e774-4a03-90e0-efd2eddb11d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:16.294455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc041712-ef8a-4943-baa7-ab567512e426 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:16.328443 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:16.339605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 398/779] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:16.339605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd618a3-4d04-423a-b322-5d625b967ccd None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:16.339605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd618a3-4d04-423a-b322-5d625b967ccd None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:16.339605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd618a3-4d04-423a-b322-5d625b967ccd None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:16.347009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88196a-8dac-43c0-84a5-136d6fd44515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:16.347765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88196a-8dac-43c0-84a5-136d6fd44515 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:16.348264 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 399/780] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:16 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:16.351211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88196a-8dac-43c0-84a5-136d6fd44515 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:16.351211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88196a-8dac-43c0-84a5-136d6fd44515 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:16.351211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88196a-8dac-43c0-84a5-136d6fd44515 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:16.351211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88196a-8dac-43c0-84a5-136d6fd44515 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:16.351738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd618a3-4d04-423a-b322-5d625b967ccd None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:16.351887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88196a-8dac-43c0-84a5-136d6fd44515 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:16.373227 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 382/781] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:16.374824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24000-9849-4feb-872f-e4890c72a7bd None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:16.375151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24000-9849-4feb-872f-e4890c72a7bd None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:16.375931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24000-9849-4feb-872f-e4890c72a7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:16.397234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d8f24000-9849-4feb-872f-e4890c72a7bd None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:16.399679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d8f24000-9849-4feb-872f-e4890c72a7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:16.492424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:16.517385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 383/782] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:16.519054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f24000-9849-4feb-872f-e4890c72a7bd None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:16.553095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd618a3-4d04-423a-b322-5d625b967ccd None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:16.667909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d51fe-2630-49b7-9a9b-8eee50a97c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:16.669015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d51fe-2630-49b7-9a9b-8eee50a97c47 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:16.669521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d51fe-2630-49b7-9a9b-8eee50a97c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:16.670014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d51fe-2630-49b7-9a9b-8eee50a97c47 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:16.676032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d51fe-2630-49b7-9a9b-8eee50a97c47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:16.677478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d51fe-2630-49b7-9a9b-8eee50a97c47 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:16.691443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 400/783] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Sep 23 01:07:16 2022] GET /identity/v3/services => generated 772 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:16.728354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d51fe-2630-49b7-9a9b-8eee50a97c47 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:16.752728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f24000-9849-4feb-872f-e4890c72a7bd None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:16.783729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:16.851895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d53556-dcc6-42bb-86f2-809ea0e18e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:16.852638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d53556-dcc6-42bb-86f2-809ea0e18e4c None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:16.853061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d53556-dcc6-42bb-86f2-809ea0e18e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:16.853344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d53556-dcc6-42bb-86f2-809ea0e18e4c None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/1c4da5b2130d4fa990adb72f0f6ec2fe/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:16.855471 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d53556-dcc6-42bb-86f2-809ea0e18e4c None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=1c4da5b2130d4fa990adb72f0f6ec2fe, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:16.857195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d53556-dcc6-42bb-86f2-809ea0e18e4c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:16.917801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d51fe-2630-49b7-9a9b-8eee50a97c47 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:16.957915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:16.962799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 384/784] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:07:16 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/1c4da5b2130d4fa990adb72f0f6ec2fe/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:07:16.964315 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d53556-dcc6-42bb-86f2-809ea0e18e4c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:16.969181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecd2d5-befe-43fb-9087-b6abfc2a2594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:16.969181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecd2d5-befe-43fb-9087-b6abfc2a2594 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:16.969181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecd2d5-befe-43fb-9087-b6abfc2a2594 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:16.969181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceecd2d5-befe-43fb-9087-b6abfc2a2594 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:16.993639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceecd2d5-befe-43fb-9087-b6abfc2a2594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-ceecd2d5-befe-43fb-9087-b6abfc2a2594 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:17.005921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:17.012727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:17.013585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:17.017853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 401/785] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:07:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:17.018961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecd2d5-befe-43fb-9087-b6abfc2a2594 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:17.189790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceecd2d5-befe-43fb-9087-b6abfc2a2594 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:17.236660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d53556-dcc6-42bb-86f2-809ea0e18e4c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:17.244403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798e310-c1f3-4175-9464-ab86e3fd5400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:17.245826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798e310-c1f3-4175-9464-ab86e3fd5400 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:17.246539 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798e310-c1f3-4175-9464-ab86e3fd5400 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:17.247268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798e310-c1f3-4175-9464-ab86e3fd5400 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:17.248788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798e310-c1f3-4175-9464-ab86e3fd5400 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:17.249584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798e310-c1f3-4175-9464-ab86e3fd5400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:17.252500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798e310-c1f3-4175-9464-ab86e3fd5400 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:17.271108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 402/786] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:07:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:17.281985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798e310-c1f3-4175-9464-ab86e3fd5400 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:17.320810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972a62e-2ef2-47f9-86e6-7d39bb986254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:17.321475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972a62e-2ef2-47f9-86e6-7d39bb986254 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:17.321752 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972a62e-2ef2-47f9-86e6-7d39bb986254 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:17.322028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972a62e-2ef2-47f9-86e6-7d39bb986254 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:17.325313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972a62e-2ef2-47f9-86e6-7d39bb986254 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:17.331517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972a62e-2ef2-47f9-86e6-7d39bb986254 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-9972a62e-2ef2-47f9-86e6-7d39bb986254 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:17.343585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:17.345672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:17.347287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 385/787] 10.222.0.64 () {56 vars in 1129 bytes} [Fri Sep 23 01:07:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:17.351445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972a62e-2ef2-47f9-86e6-7d39bb986254 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:17.391669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:17.445328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:17.466140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798e310-c1f3-4175-9464-ab86e3fd5400 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:17.518576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0288fc8-e4f3-4f82-a7e7-8a1b4bbc448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:17.519636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0288fc8-e4f3-4f82-a7e7-8a1b4bbc448a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:17.521156 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0288fc8-e4f3-4f82-a7e7-8a1b4bbc448a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:17.521769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0288fc8-e4f3-4f82-a7e7-8a1b4bbc448a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:17.550652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0288fc8-e4f3-4f82-a7e7-8a1b4bbc448a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:17.552669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0288fc8-e4f3-4f82-a7e7-8a1b4bbc448a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-e0288fc8-e4f3-4f82-a7e7-8a1b4bbc448a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:17.583299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:17.584861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:17.586248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:17.586248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:17.586248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:17.591061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 403/788] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:17.598081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0288fc8-e4f3-4f82-a7e7-8a1b4bbc448a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:17.638228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972a62e-2ef2-47f9-86e6-7d39bb986254 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:17.771074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc9a55-5366-4222-a236-7f515a6add53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:17.772077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc9a55-5366-4222-a236-7f515a6add53 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:17.772569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc9a55-5366-4222-a236-7f515a6add53 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:17.773037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc9a55-5366-4222-a236-7f515a6add53 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:17.774209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc9a55-5366-4222-a236-7f515a6add53 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:17.774747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc9a55-5366-4222-a236-7f515a6add53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:17.777389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc9a55-5366-4222-a236-7f515a6add53 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:17.787339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 386/789] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Sep 23 01:07:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:17.796803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc9a55-5366-4222-a236-7f515a6add53 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:17.862214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:17.866768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0288fc8-e4f3-4f82-a7e7-8a1b4bbc448a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:17.921518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62318b5f-3cbc-451e-90a5-f0b6be91c4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:17.922188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318b5f-3cbc-451e-90a5-f0b6be91c4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:17.922468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318b5f-3cbc-451e-90a5-f0b6be91c4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:17.922742 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62318b5f-3cbc-451e-90a5-f0b6be91c4f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:17.923402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62318b5f-3cbc-451e-90a5-f0b6be91c4f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:17.923737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62318b5f-3cbc-451e-90a5-f0b6be91c4f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:17.925307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62318b5f-3cbc-451e-90a5-f0b6be91c4f9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:17.948372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 404/790] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:17.960125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62318b5f-3cbc-451e-90a5-f0b6be91c4f9 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:18.021814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:18.026690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc9a55-5366-4222-a236-7f515a6add53 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:18.092430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca57ac-67e2-489d-bd4c-069dc7c82154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:18.093406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca57ac-67e2-489d-bd4c-069dc7c82154 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:18.094529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca57ac-67e2-489d-bd4c-069dc7c82154 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:18.095013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca57ac-67e2-489d-bd4c-069dc7c82154 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:18.096207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca57ac-67e2-489d-bd4c-069dc7c82154 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:18.096722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca57ac-67e2-489d-bd4c-069dc7c82154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:18.108946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca57ac-67e2-489d-bd4c-069dc7c82154 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:18.130825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 387/791] 10.222.0.64 () {56 vars in 1124 bytes} [Fri Sep 23 01:07:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:18.155776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca57ac-67e2-489d-bd4c-069dc7c82154 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:18.261165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62318b5f-3cbc-451e-90a5-f0b6be91c4f9 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:18.331454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb53147-c128-4596-8055-4cf16bce2cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:18.332049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb53147-c128-4596-8055-4cf16bce2cfb None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:18.332336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb53147-c128-4596-8055-4cf16bce2cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:18.332634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb53147-c128-4596-8055-4cf16bce2cfb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:18.360824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb53147-c128-4596-8055-4cf16bce2cfb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:18.362498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb53147-c128-4596-8055-4cf16bce2cfb None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-efb53147-c128-4596-8055-4cf16bce2cfb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:18.389935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:18.392779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:18.394835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 405/792] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:07:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:18.404885 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb53147-c128-4596-8055-4cf16bce2cfb None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:18.456778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca57ac-67e2-489d-bd4c-069dc7c82154 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:18.478661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:18.556017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d3c97-82cb-432b-8d50-819b6189fcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:18.557572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d3c97-82cb-432b-8d50-819b6189fcc9 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:18.558372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d3c97-82cb-432b-8d50-819b6189fcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:18.559159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d3c97-82cb-432b-8d50-819b6189fcc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:18.560753 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d3c97-82cb-432b-8d50-819b6189fcc9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:18.564564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d3c97-82cb-432b-8d50-819b6189fcc9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:18.650457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb53147-c128-4596-8055-4cf16bce2cfb None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:18.721146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6fb6a-37aa-416c-9e7c-2986d60566d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:18.722226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6fb6a-37aa-416c-9e7c-2986d60566d6 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:18.722706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6fb6a-37aa-416c-9e7c-2986d60566d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:18.723177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6fb6a-37aa-416c-9e7c-2986d60566d6 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:18.724710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6fb6a-37aa-416c-9e7c-2986d60566d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:18.725259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6fb6a-37aa-416c-9e7c-2986d60566d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:18.728267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6fb6a-37aa-416c-9e7c-2986d60566d6 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:18.739573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 388/793] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:07:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 585 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:07:18.767154 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 406/794] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Sep 23 01:07:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:18.775734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d3c97-82cb-432b-8d50-819b6189fcc9 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:18.857342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:19.051057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d3c97-82cb-432b-8d50-819b6189fcc9 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:19.129427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27726344-6619-4ed8-922d-7d7418b45aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:19.133398 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27726344-6619-4ed8-922d-7d7418b45aaa None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:19.134470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27726344-6619-4ed8-922d-7d7418b45aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:19.135242 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27726344-6619-4ed8-922d-7d7418b45aaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:19.155949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27726344-6619-4ed8-922d-7d7418b45aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:19.159196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27726344-6619-4ed8-922d-7d7418b45aaa None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-27726344-6619-4ed8-922d-7d7418b45aaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:19.192250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:19.194347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:19.196085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:19.196085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:19.196085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:19.204074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 389/795] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:19.211201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6fb6a-37aa-416c-9e7c-2986d60566d6 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:19.248872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72777a-a49f-4859-a805-e6e6f1b9e489 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:19.249124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72777a-a49f-4859-a805-e6e6f1b9e489 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:19.249453 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72777a-a49f-4859-a805-e6e6f1b9e489 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:19.251969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 390/796] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:07:19.263254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6099d3-e81a-4b9e-8dba-b5aa5ac0edb0 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:19.263866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6099d3-e81a-4b9e-8dba-b5aa5ac0edb0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:19.264306 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6099d3-e81a-4b9e-8dba-b5aa5ac0edb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:19.299582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-3a6099d3-e81a-4b9e-8dba-b5aa5ac0edb0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:19.301982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3a6099d3-e81a-4b9e-8dba-b5aa5ac0edb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:19.327489 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:19.353589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6fb6a-37aa-416c-9e7c-2986d60566d6 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:19.412532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb536c7-6bb8-4b99-b10a-d18d45d1e51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:19.413167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb536c7-6bb8-4b99-b10a-d18d45d1e51c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:19.413446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb536c7-6bb8-4b99-b10a-d18d45d1e51c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:19.413728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb536c7-6bb8-4b99-b10a-d18d45d1e51c None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:19.414429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb536c7-6bb8-4b99-b10a-d18d45d1e51c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:19.414692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb536c7-6bb8-4b99-b10a-d18d45d1e51c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:19.416222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb536c7-6bb8-4b99-b10a-d18d45d1e51c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:19.431326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 407/797] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:19.432200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:19.437227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb536c7-6bb8-4b99-b10a-d18d45d1e51c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:19.451590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 391/798] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:19.572230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb536c7-6bb8-4b99-b10a-d18d45d1e51c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:19.654317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51847aba-e9ed-4fdd-9ac4-845ee30b8f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:19.655394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51847aba-e9ed-4fdd-9ac4-845ee30b8f45 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:19.655714 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51847aba-e9ed-4fdd-9ac4-845ee30b8f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:19.656053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51847aba-e9ed-4fdd-9ac4-845ee30b8f45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:19.671120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51847aba-e9ed-4fdd-9ac4-845ee30b8f45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:19.674162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51847aba-e9ed-4fdd-9ac4-845ee30b8f45 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:19.677830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-51847aba-e9ed-4fdd-9ac4-845ee30b8f45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:19.695590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:19.696653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:19.697557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:19.697557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:19.697557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:19.697557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:19.697557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:19.697557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:19.697557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:19.697557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:19.697557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 408/799] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:19.705698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6099d3-e81a-4b9e-8dba-b5aa5ac0edb0 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:19.876004 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6099d3-e81a-4b9e-8dba-b5aa5ac0edb0 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:19.931022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc6e9-8bb6-4d2f-b23f-07def258a14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:19.931771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc6e9-8bb6-4d2f-b23f-07def258a14a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:19.932055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc6e9-8bb6-4d2f-b23f-07def258a14a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:19.932334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdc6e9-8bb6-4d2f-b23f-07def258a14a None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:19.933043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc6e9-8bb6-4d2f-b23f-07def258a14a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:19.933360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc6e9-8bb6-4d2f-b23f-07def258a14a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:19.935109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdc6e9-8bb6-4d2f-b23f-07def258a14a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:19.939342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803d1af-f454-42f5-9a23-6455a81ed987 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:19.940091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803d1af-f454-42f5-9a23-6455a81ed987 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:19.940592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803d1af-f454-42f5-9a23-6455a81ed987 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:19.943865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 409/800] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:19.951220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 392/801] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:19.952300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde1f16-8ef3-4a07-ab57-89466af86c0e None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:19.952585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde1f16-8ef3-4a07-ab57-89466af86c0e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:19.952883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde1f16-8ef3-4a07-ab57-89466af86c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:19.959859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc6e9-8bb6-4d2f-b23f-07def258a14a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:19.982082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-8dde1f16-8ef3-4a07-ab57-89466af86c0e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:19.983539 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-8dde1f16-8ef3-4a07-ab57-89466af86c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:20.052047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:20.111950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 410/802] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:20.113362 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde1f16-8ef3-4a07-ab57-89466af86c0e None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:20.119450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdc6e9-8bb6-4d2f-b23f-07def258a14a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:20.207070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec43e8a-3a3f-467f-8d18-d4874b9ed305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:20.208134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec43e8a-3a3f-467f-8d18-d4874b9ed305 None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:20.208684 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec43e8a-3a3f-467f-8d18-d4874b9ed305 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:20.208971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec43e8a-3a3f-467f-8d18-d4874b9ed305 None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/c5a9f3eca3e34fffa7b218064e831d7b/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:20.214285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec43e8a-3a3f-467f-8d18-d4874b9ed305 None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=c5a9f3eca3e34fffa7b218064e831d7b, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:20.216031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec43e8a-3a3f-467f-8d18-d4874b9ed305 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:20.317968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 393/803] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:07:19 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/c5a9f3eca3e34fffa7b218064e831d7b/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:07:20.323701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dde1f16-8ef3-4a07-ab57-89466af86c0e None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:20.325391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e8e7-9cc3-4e9a-8480-75d81c03548c None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:20.329825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e8e7-9cc3-4e9a-8480-75d81c03548c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:20.330638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e8e7-9cc3-4e9a-8480-75d81c03548c None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:20.334631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 394/804] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:20 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:20.342499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f6ab3-25b1-41bf-9298-a259bcef274f None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:20.343216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f6ab3-25b1-41bf-9298-a259bcef274f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:20.347513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f6ab3-25b1-41bf-9298-a259bcef274f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:20.404626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-0f6f6ab3-25b1-41bf-9298-a259bcef274f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:20.408823 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0f6f6ab3-25b1-41bf-9298-a259bcef274f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:20.412923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:20.437729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ac31c-94af-49e2-a668-75b1004c0622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:20.438409 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ac31c-94af-49e2-a668-75b1004c0622 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:20.438694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ac31c-94af-49e2-a668-75b1004c0622 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:20.438967 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ac31c-94af-49e2-a668-75b1004c0622 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:20.483755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ac31c-94af-49e2-a668-75b1004c0622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-562ac31c-94af-49e2-a668-75b1004c0622 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:20.506187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:20.508448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:07:20.510454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:20.513454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 411/805] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:07:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:20.521155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ac31c-94af-49e2-a668-75b1004c0622 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:20.590599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 395/806] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:20.631715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f6ab3-25b1-41bf-9298-a259bcef274f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:20.722691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:20.753715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ac31c-94af-49e2-a668-75b1004c0622 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:20.823377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f6ab3-25b1-41bf-9298-a259bcef274f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:20.840870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0946a0-eb04-4ba6-89e2-a9a770a0eaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:20.841069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0946a0-eb04-4ba6-89e2-a9a770a0eaf3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:20.841227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0946a0-eb04-4ba6-89e2-a9a770a0eaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:20.841503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0946a0-eb04-4ba6-89e2-a9a770a0eaf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:20.842206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0946a0-eb04-4ba6-89e2-a9a770a0eaf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:20.842511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0946a0-eb04-4ba6-89e2-a9a770a0eaf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:20.844719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0946a0-eb04-4ba6-89e2-a9a770a0eaf3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:20.863467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 412/807] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:07:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:20.870104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0946a0-eb04-4ba6-89e2-a9a770a0eaf3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:20.878373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ab37-dff7-47d9-a691-d8ef4d9558fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:20.881260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ab37-dff7-47d9-a691-d8ef4d9558fe None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:20.881781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ab37-dff7-47d9-a691-d8ef4d9558fe None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:20.882271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6ab37-dff7-47d9-a691-d8ef4d9558fe None admin] PATH_INFO: `/v3/services/network` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:20.883170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6ab37-dff7-47d9-a691-d8ef4d9558fe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:20.885056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6ab37-dff7-47d9-a691-d8ef4d9558fe None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-39b6ab37-dff7-47d9-a691-d8ef4d9558fe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:20.905382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:20.907141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 23 01:07:20.908027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:20.911769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 396/808] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:20 2022] GET /identity/v3/services/network => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:20.921559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ab37-dff7-47d9-a691-d8ef4d9558fe None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:20.971132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:21.059523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0946a0-eb04-4ba6-89e2-a9a770a0eaf3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:21.096224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6ab37-dff7-47d9-a691-d8ef4d9558fe None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:21.121868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:21.128347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66ff72-b540-4d42-8023-84b9635e1cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:21.130132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66ff72-b540-4d42-8023-84b9635e1cdd None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:21.131003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66ff72-b540-4d42-8023-84b9635e1cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:21.131859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66ff72-b540-4d42-8023-84b9635e1cdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:21.151414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e62215-85ef-4c60-9008-a5c4b359f9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:21.151942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e62215-85ef-4c60-9008-a5c4b359f9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:21.152280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e62215-85ef-4c60-9008-a5c4b359f9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:21.152670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e62215-85ef-4c60-9008-a5c4b359f9e8 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:21.153340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e62215-85ef-4c60-9008-a5c4b359f9e8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:21.153645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e62215-85ef-4c60-9008-a5c4b359f9e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:21.155634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e62215-85ef-4c60-9008-a5c4b359f9e8 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:21.158093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66ff72-b540-4d42-8023-84b9635e1cdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:21.161670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66ff72-b540-4d42-8023-84b9635e1cdd None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:21.166826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 397/809] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:21.169507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e62215-85ef-4c60-9008-a5c4b359f9e8 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-8e66ff72-b540-4d42-8023-84b9635e1cdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:21.200555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:21.201589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:21.202471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:21.202471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:21.202471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:21.203787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 413/810] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:21.205262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66ff72-b540-4d42-8023-84b9635e1cdd None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:21.307170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e62215-85ef-4c60-9008-a5c4b359f9e8 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:21.350066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455285c-4a6c-4d01-834d-504ece4d2ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:21.351527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455285c-4a6c-4d01-834d-504ece4d2ecf None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:21.352337 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455285c-4a6c-4d01-834d-504ece4d2ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:21.353055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455285c-4a6c-4d01-834d-504ece4d2ecf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:21.372988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455285c-4a6c-4d01-834d-504ece4d2ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-0455285c-4a6c-4d01-834d-504ece4d2ecf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:21.384438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:21.386237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:21.387816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:21.390026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66ff72-b540-4d42-8023-84b9635e1cdd None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:21.392474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:21.393680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 398/811] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:07:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:21.395433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455285c-4a6c-4d01-834d-504ece4d2ecf None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:21.436835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0099e-2b07-4f14-b953-50025e88240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:21.438149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0099e-2b07-4f14-b953-50025e88240b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:21.438441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0099e-2b07-4f14-b953-50025e88240b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:21.438718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0099e-2b07-4f14-b953-50025e88240b None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:21.439386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0099e-2b07-4f14-b953-50025e88240b None admin] RBAC: Adding query filter params (type=network) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:21.439846 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0099e-2b07-4f14-b953-50025e88240b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:21.442323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0099e-2b07-4f14-b953-50025e88240b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:21.455824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 414/812] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:21.455824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0099e-2b07-4f14-b953-50025e88240b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:21.477852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:21.582272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0099e-2b07-4f14-b953-50025e88240b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:21.611199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455285c-4a6c-4d01-834d-504ece4d2ecf None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:21.626988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc3b66-c11f-4108-b290-071bdbbcbace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:21.627815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc3b66-c11f-4108-b290-071bdbbcbace None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:21.628204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc3b66-c11f-4108-b290-071bdbbcbace None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:21.628583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cc3b66-c11f-4108-b290-071bdbbcbace None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:21.629402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc3b66-c11f-4108-b290-071bdbbcbace None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:21.629851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc3b66-c11f-4108-b290-071bdbbcbace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:21.631820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cc3b66-c11f-4108-b290-071bdbbcbace None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:21.646756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 415/813] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:07:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:21.648274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc3b66-c11f-4108-b290-071bdbbcbace None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:21.695593 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:21.833868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cc3b66-c11f-4108-b290-071bdbbcbace None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:21.857877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924dc88c-9253-41c0-87a8-38380f326fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:21.859655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dc88c-9253-41c0-87a8-38380f326fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:21.860796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dc88c-9253-41c0-87a8-38380f326fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:21.861369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dc88c-9253-41c0-87a8-38380f326fc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:21.863097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924dc88c-9253-41c0-87a8-38380f326fc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:21.863732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924dc88c-9253-41c0-87a8-38380f326fc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:21.866838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924dc88c-9253-41c0-87a8-38380f326fc3 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:21.891618 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c24e3-5f35-4289-bf40-08ab7b91de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:21.892659 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c24e3-5f35-4289-bf40-08ab7b91de8d None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:21.893184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c24e3-5f35-4289-bf40-08ab7b91de8d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:21.893689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c24e3-5f35-4289-bf40-08ab7b91de8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:21.894811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 399/814] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:21.896100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c24e3-5f35-4289-bf40-08ab7b91de8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:21.897911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c24e3-5f35-4289-bf40-08ab7b91de8d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:21.900543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924dc88c-9253-41c0-87a8-38380f326fc3 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:22.000894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 416/815] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:07:21 2022] POST /identity/v3/endpoints => generated 337 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:07:22.003143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c24e3-5f35-4289-bf40-08ab7b91de8d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:22.143534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:22.168878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924dc88c-9253-41c0-87a8-38380f326fc3 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:22.259925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:22.303682 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b951e5b-2fb1-44d3-9a74-fddba82079e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:22.304569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b951e5b-2fb1-44d3-9a74-fddba82079e1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:22.304893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b951e5b-2fb1-44d3-9a74-fddba82079e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:22.305168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b951e5b-2fb1-44d3-9a74-fddba82079e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:22.307677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c24e3-5f35-4289-bf40-08ab7b91de8d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:22.324992 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b951e5b-2fb1-44d3-9a74-fddba82079e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:22.326589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b951e5b-2fb1-44d3-9a74-fddba82079e1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-4b951e5b-2fb1-44d3-9a74-fddba82079e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:22.353141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:22.355347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:22.357519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:22.357519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:22.357519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:22.357519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 400/816] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:22.363371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b951e5b-2fb1-44d3-9a74-fddba82079e1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:22.405802 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76743819-3a6c-47a5-bcd3-84fb0d4a3b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:22.407994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76743819-3a6c-47a5-bcd3-84fb0d4a3b31 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:22.408572 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76743819-3a6c-47a5-bcd3-84fb0d4a3b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:22.409041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76743819-3a6c-47a5-bcd3-84fb0d4a3b31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:22.419002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76743819-3a6c-47a5-bcd3-84fb0d4a3b31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:22.420894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76743819-3a6c-47a5-bcd3-84fb0d4a3b31 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-76743819-3a6c-47a5-bcd3-84fb0d4a3b31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:22.437012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:22.438457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:22.439686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:22.439686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:22.439686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:22.439686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:22.439686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:22.439686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:22.439686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:22.439686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:22.439686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:22.439686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:22.443286 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 417/817] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Sep 23 01:07:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:22.450893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76743819-3a6c-47a5-bcd3-84fb0d4a3b31 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:22.536152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b951e5b-2fb1-44d3-9a74-fddba82079e1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:22.602192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76743819-3a6c-47a5-bcd3-84fb0d4a3b31 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:22.643667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3021b-6b45-4756-aea2-939490175a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:22.644980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3021b-6b45-4756-aea2-939490175a65 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:22.645612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3021b-6b45-4756-aea2-939490175a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:22.646197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3021b-6b45-4756-aea2-939490175a65 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:22.647663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3021b-6b45-4756-aea2-939490175a65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:22.647825 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b4713-ef74-46b9-ab57-95bbbe34bdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:22.648347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3021b-6b45-4756-aea2-939490175a65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:22.648459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b4713-ef74-46b9-ab57-95bbbe34bdc1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:22.648647 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b4713-ef74-46b9-ab57-95bbbe34bdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:22.648925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3b4713-ef74-46b9-ab57-95bbbe34bdc1 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:22.649705 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b4713-ef74-46b9-ab57-95bbbe34bdc1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:22.650014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b4713-ef74-46b9-ab57-95bbbe34bdc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:22.650967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3021b-6b45-4756-aea2-939490175a65 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:22.651471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3b4713-ef74-46b9-ab57-95bbbe34bdc1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:22.663383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:22.674315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 418/818] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Sep 23 01:07:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:22.680577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b4713-ef74-46b9-ab57-95bbbe34bdc1 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:22.683886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 401/819] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:22.688981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3021b-6b45-4756-aea2-939490175a65 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:22.708195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:22.847206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3b4713-ef74-46b9-ab57-95bbbe34bdc1 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:22.892183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3021b-6b45-4756-aea2-939490175a65 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:22.898449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9d9780-2ee8-42f2-b101-420da83cc07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:22.899474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d9780-2ee8-42f2-b101-420da83cc07b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:22.899868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d9780-2ee8-42f2-b101-420da83cc07b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:22.900216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9d9780-2ee8-42f2-b101-420da83cc07b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:22.918342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d9780-2ee8-42f2-b101-420da83cc07b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:22.920440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9d9780-2ee8-42f2-b101-420da83cc07b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-ef9d9780-2ee8-42f2-b101-420da83cc07b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:22.941525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:22.944043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:22.946238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:22.946238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:22.946238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:22.946990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 419/820] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:22.949727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd981e4-3902-4883-95a5-c54246dbd1fb None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:22.950201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd981e4-3902-4883-95a5-c54246dbd1fb None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:22.950691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd981e4-3902-4883-95a5-c54246dbd1fb None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:22.960716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 420/821] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:22.960716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd981e4-3902-4883-95a5-c54246dbd1fb None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:22.966911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55261267-198e-49fb-b906-4043e8340690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:22.967973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55261267-198e-49fb-b906-4043e8340690 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:22.968942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55261267-198e-49fb-b906-4043e8340690 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:22.969392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55261267-198e-49fb-b906-4043e8340690 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:22.981296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55261267-198e-49fb-b906-4043e8340690 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:22.983356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55261267-198e-49fb-b906-4043e8340690 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-55261267-198e-49fb-b906-4043e8340690 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:22.999636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:23.010462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 402/822] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Sep 23 01:07:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53717f1-1511-4b68-a205-e69b5810a788 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:23.012777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53717f1-1511-4b68-a205-e69b5810a788 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:23.014174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53717f1-1511-4b68-a205-e69b5810a788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:23.038500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d53717f1-1511-4b68-a205-e69b5810a788 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:23.040599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d53717f1-1511-4b68-a205-e69b5810a788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:23.087675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:23.139974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:23.155611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 403/823] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:23.155611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53717f1-1511-4b68-a205-e69b5810a788 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:23.200219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd981e4-3902-4883-95a5-c54246dbd1fb None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:23.254395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2813d8b-3d53-47f1-8a25-7ab5f9a5741b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:23.255216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2813d8b-3d53-47f1-8a25-7ab5f9a5741b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:23.255849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2813d8b-3d53-47f1-8a25-7ab5f9a5741b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:23.256289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2813d8b-3d53-47f1-8a25-7ab5f9a5741b None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:23.257110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2813d8b-3d53-47f1-8a25-7ab5f9a5741b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:23.257574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2813d8b-3d53-47f1-8a25-7ab5f9a5741b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:23.259811 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2813d8b-3d53-47f1-8a25-7ab5f9a5741b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:23.282161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 421/824] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:23.282161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13e990-d4aa-4a10-95b0-2219082d6230 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:23.282161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13e990-d4aa-4a10-95b0-2219082d6230 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:23.282161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13e990-d4aa-4a10-95b0-2219082d6230 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:23.282161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 422/825] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:23.284222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13e990-d4aa-4a10-95b0-2219082d6230 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:23.329140 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53717f1-1511-4b68-a205-e69b5810a788 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:23.372527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef8d8b-7efe-4190-8f7f-25c9efe9b570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:23.373148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef8d8b-7efe-4190-8f7f-25c9efe9b570 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:23.373418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef8d8b-7efe-4190-8f7f-25c9efe9b570 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:23.373696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef8d8b-7efe-4190-8f7f-25c9efe9b570 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:23.374490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef8d8b-7efe-4190-8f7f-25c9efe9b570 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:23.374829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef8d8b-7efe-4190-8f7f-25c9efe9b570 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:23.376317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef8d8b-7efe-4190-8f7f-25c9efe9b570 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:23.398579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 404/826] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Sep 23 01:07:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:23.399737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c704a-4289-4c3a-a661-b76be6795fae None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:23.400108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c704a-4289-4c3a-a661-b76be6795fae None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:23.400392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c704a-4289-4c3a-a661-b76be6795fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:23.422892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d63c704a-4289-4c3a-a661-b76be6795fae None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:23.423198 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d63c704a-4289-4c3a-a661-b76be6795fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:23.490348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13e990-d4aa-4a10-95b0-2219082d6230 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:23.492432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:23.508624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:23.519949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 405/827] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:23 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:23.520831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c704a-4289-4c3a-a661-b76be6795fae None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:23.544288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f614b4d-0c8c-4f6d-bb18-029a1cec5f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:23.546269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f614b4d-0c8c-4f6d-bb18-029a1cec5f55 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:23.546269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f614b4d-0c8c-4f6d-bb18-029a1cec5f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:23.546269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f614b4d-0c8c-4f6d-bb18-029a1cec5f55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:23.559160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f614b4d-0c8c-4f6d-bb18-029a1cec5f55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:23.561995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f614b4d-0c8c-4f6d-bb18-029a1cec5f55 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-5f614b4d-0c8c-4f6d-bb18-029a1cec5f55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:23.585963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:23.588250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:23.590033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:23.590033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:23.590033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:23.590033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:23.590033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:23.590033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:23.590033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:23.590033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:23.590033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 423/828] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:23.590993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f614b4d-0c8c-4f6d-bb18-029a1cec5f55 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:23.664312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c704a-4289-4c3a-a661-b76be6795fae None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:23.724749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff726836-7086-4758-a9fb-b7ce66af9014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:23.726022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff726836-7086-4758-a9fb-b7ce66af9014 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:23.726520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff726836-7086-4758-a9fb-b7ce66af9014 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:23.726989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff726836-7086-4758-a9fb-b7ce66af9014 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:23.736559 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f614b4d-0c8c-4f6d-bb18-029a1cec5f55 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:23.784900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3285b7-c1cd-4f5f-a26a-50a698ea3340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:23.785443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3285b7-c1cd-4f5f-a26a-50a698ea3340 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:23.785727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3285b7-c1cd-4f5f-a26a-50a698ea3340 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:23.786020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3285b7-c1cd-4f5f-a26a-50a698ea3340 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:23.786598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3285b7-c1cd-4f5f-a26a-50a698ea3340 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:23.788044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3285b7-c1cd-4f5f-a26a-50a698ea3340 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-3d3285b7-c1cd-4f5f-a26a-50a698ea3340 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:23.798939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:23.801146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:23.804543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 424/829] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:07:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:23.807679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3285b7-c1cd-4f5f-a26a-50a698ea3340 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:23.813151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:23.834924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff726836-7086-4758-a9fb-b7ce66af9014 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:23.836551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff726836-7086-4758-a9fb-b7ce66af9014 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:23.858692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:23.946420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3285b7-c1cd-4f5f-a26a-50a698ea3340 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:23.971160 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 406/830] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:07:23 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:07:23.972211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff726836-7086-4758-a9fb-b7ce66af9014 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:24.004320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f9d99-a8f5-4308-a541-21f8142026e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:24.004980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f9d99-a8f5-4308-a541-21f8142026e8 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:24.005393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f9d99-a8f5-4308-a541-21f8142026e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:24.005664 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f9d99-a8f5-4308-a541-21f8142026e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:24.006511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f9d99-a8f5-4308-a541-21f8142026e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:24.006816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f9d99-a8f5-4308-a541-21f8142026e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:24.008781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f9d99-a8f5-4308-a541-21f8142026e8 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:24.033229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 425/831] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:24.033229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f9d99-a8f5-4308-a541-21f8142026e8 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:24.105285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:24.136410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff726836-7086-4758-a9fb-b7ce66af9014 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:24.160878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:24.169623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f9d99-a8f5-4308-a541-21f8142026e8 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:24.207238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ca19b-030a-4198-8a2f-beb381b7621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:24.208321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ca19b-030a-4198-8a2f-beb381b7621e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:24.208804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ca19b-030a-4198-8a2f-beb381b7621e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:24.209271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ca19b-030a-4198-8a2f-beb381b7621e None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:24.210336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ca19b-030a-4198-8a2f-beb381b7621e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:24.210881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ca19b-030a-4198-8a2f-beb381b7621e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:24.213474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ca19b-030a-4198-8a2f-beb381b7621e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:24.213844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f40702-f5c0-49c1-8b63-91c216feaa32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:24.214620 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f40702-f5c0-49c1-8b63-91c216feaa32 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:24.214958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f40702-f5c0-49c1-8b63-91c216feaa32 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:24.215292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f40702-f5c0-49c1-8b63-91c216feaa32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:24.227880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 407/832] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:07:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:24.227880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ca19b-030a-4198-8a2f-beb381b7621e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:24.228693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f40702-f5c0-49c1-8b63-91c216feaa32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:24.230453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f40702-f5c0-49c1-8b63-91c216feaa32 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-f3f40702-f5c0-49c1-8b63-91c216feaa32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:24.248134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:24.251346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:24.253105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:24.253105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:24.253105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:24.253105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 426/833] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:24.253105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f40702-f5c0-49c1-8b63-91c216feaa32 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:24.366947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ca19b-030a-4198-8a2f-beb381b7621e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:24.391661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f40702-f5c0-49c1-8b63-91c216feaa32 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:24.424811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1197db8-19cc-4c86-bcac-6cdf09515a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:24.425500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1197db8-19cc-4c86-bcac-6cdf09515a3b None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:24.425798 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1197db8-19cc-4c86-bcac-6cdf09515a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:24.426070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1197db8-19cc-4c86-bcac-6cdf09515a3b None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/fa998fcd51b3404aab5412323f86908f/roles/50ceaff2140f4e05a56877fee2f609e4` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:24.431953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313444c8-f715-4700-bbe9-dba635232e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:24.432983 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313444c8-f715-4700-bbe9-dba635232e82 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:24.433461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313444c8-f715-4700-bbe9-dba635232e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:24.433961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313444c8-f715-4700-bbe9-dba635232e82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:24.443454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1197db8-19cc-4c86-bcac-6cdf09515a3b None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=fa998fcd51b3404aab5412323f86908f, role_id=50ceaff2140f4e05a56877fee2f609e4)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:24.445194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1197db8-19cc-4c86-bcac-6cdf09515a3b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:24.448674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:24.457774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313444c8-f715-4700-bbe9-dba635232e82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-313444c8-f715-4700-bbe9-dba635232e82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:24.463541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:24.465229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 427/834] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:07:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:24.466674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313444c8-f715-4700-bbe9-dba635232e82 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:24.477346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:24.564087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 408/835] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:07:24 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/fa998fcd51b3404aab5412323f86908f/roles/50ceaff2140f4e05a56877fee2f609e4 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:07:24.565480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1197db8-19cc-4c86-bcac-6cdf09515a3b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:24.581172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313444c8-f715-4700-bbe9-dba635232e82 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:24.643676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428ad31-545b-4633-9967-7cdd46d4c48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:24.644645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428ad31-545b-4633-9967-7cdd46d4c48d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:24.645146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428ad31-545b-4633-9967-7cdd46d4c48d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:24.645645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428ad31-545b-4633-9967-7cdd46d4c48d None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:24.646794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428ad31-545b-4633-9967-7cdd46d4c48d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:24.647334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428ad31-545b-4633-9967-7cdd46d4c48d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:24.650192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428ad31-545b-4633-9967-7cdd46d4c48d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:24.672019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 428/836] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:07:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:24.672019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428ad31-545b-4633-9967-7cdd46d4c48d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:24.780369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:24.786925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1197db8-19cc-4c86-bcac-6cdf09515a3b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:24.827971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428ad31-545b-4633-9967-7cdd46d4c48d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:24.892290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cd3a4-739a-4371-bb59-dc234eb5acdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:24.893903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cd3a4-739a-4371-bb59-dc234eb5acdd None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:24.894643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cd3a4-739a-4371-bb59-dc234eb5acdd None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:24.895363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cd3a4-739a-4371-bb59-dc234eb5acdd None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:24.897068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cd3a4-739a-4371-bb59-dc234eb5acdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:24.897894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cd3a4-739a-4371-bb59-dc234eb5acdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:24.898092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469e91b-958b-426f-8e83-b4fe22552f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:24.898952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469e91b-958b-426f-8e83-b4fe22552f2a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:24.899382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469e91b-958b-426f-8e83-b4fe22552f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:24.899836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469e91b-958b-426f-8e83-b4fe22552f2a None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:24.900897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469e91b-958b-426f-8e83-b4fe22552f2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:24.901354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469e91b-958b-426f-8e83-b4fe22552f2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:24.902333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cd3a4-739a-4371-bb59-dc234eb5acdd None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:24.904417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469e91b-958b-426f-8e83-b4fe22552f2a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:24.924906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 429/837] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:07:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:24.927529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 409/838] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:24.932444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469e91b-958b-426f-8e83-b4fe22552f2a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:24.936861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cd3a4-739a-4371-bb59-dc234eb5acdd None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:25.055645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:25.151483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cd3a4-739a-4371-bb59-dc234eb5acdd None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:25.158905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469e91b-958b-426f-8e83-b4fe22552f2a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:25.221884 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:25.222530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78615133-d4ec-4c73-b530-800489cf73f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:25.225047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78615133-d4ec-4c73-b530-800489cf73f7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:25.225768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78615133-d4ec-4c73-b530-800489cf73f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:25.226612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78615133-d4ec-4c73-b530-800489cf73f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:25.229959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0333e7-df5e-4d73-ba4c-53bcbebaa1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:25.231070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0333e7-df5e-4d73-ba4c-53bcbebaa1ea None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:25.231676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0333e7-df5e-4d73-ba4c-53bcbebaa1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:25.240216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0333e7-df5e-4d73-ba4c-53bcbebaa1ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:25.242737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78615133-d4ec-4c73-b530-800489cf73f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:25.245613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78615133-d4ec-4c73-b530-800489cf73f7 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:25.258169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0333e7-df5e-4d73-ba4c-53bcbebaa1ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:25.261283 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0333e7-df5e-4d73-ba4c-53bcbebaa1ea None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-78615133-d4ec-4c73-b530-800489cf73f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:25.270056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:25.272110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:25.274125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:25.274125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:25.274125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:25.274125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:25.274125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:25.274125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:25.274125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:25.274125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:25.274125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 410/839] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:25.282991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78615133-d4ec-4c73-b530-800489cf73f7 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-0e0333e7-df5e-4d73-ba4c-53bcbebaa1ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:25.302009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:25.303853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:25.305695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:25.305695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:25.305695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:25.307746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 430/840] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:25.318103 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0333e7-df5e-4d73-ba4c-53bcbebaa1ea None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:25.474467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0333e7-df5e-4d73-ba4c-53bcbebaa1ea None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:25.512542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78615133-d4ec-4c73-b530-800489cf73f7 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:25.534549 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:25.557993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074df88-9747-461d-8d79-df9414ef64bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:25.558648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074df88-9747-461d-8d79-df9414ef64bc None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:25.558935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074df88-9747-461d-8d79-df9414ef64bc None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:25.559209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074df88-9747-461d-8d79-df9414ef64bc None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:25.560112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074df88-9747-461d-8d79-df9414ef64bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:25.560426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074df88-9747-461d-8d79-df9414ef64bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:25.560930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534c097-0b1b-41a9-8fbd-f990b51eef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:25.562135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074df88-9747-461d-8d79-df9414ef64bc None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:25.562868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534c097-0b1b-41a9-8fbd-f990b51eef59 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:25.563752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534c097-0b1b-41a9-8fbd-f990b51eef59 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:25.564554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534c097-0b1b-41a9-8fbd-f990b51eef59 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:25.566122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534c097-0b1b-41a9-8fbd-f990b51eef59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:25.566985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534c097-0b1b-41a9-8fbd-f990b51eef59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:25.570490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534c097-0b1b-41a9-8fbd-f990b51eef59 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:25.579938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 411/841] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:25.589939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074df88-9747-461d-8d79-df9414ef64bc None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:25.601135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 431/842] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:25.610799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534c097-0b1b-41a9-8fbd-f990b51eef59 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:25.669193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:25.746135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534c097-0b1b-41a9-8fbd-f990b51eef59 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:25.790845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e65dce-bae0-40a6-8b4a-582a2f2f457a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:25.791600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074df88-9747-461d-8d79-df9414ef64bc None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:25.791970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e65dce-bae0-40a6-8b4a-582a2f2f457a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:25.792419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e65dce-bae0-40a6-8b4a-582a2f2f457a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:25.792887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e65dce-bae0-40a6-8b4a-582a2f2f457a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:25.802528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e65dce-bae0-40a6-8b4a-582a2f2f457a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:25.805109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e65dce-bae0-40a6-8b4a-582a2f2f457a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-c0e65dce-bae0-40a6-8b4a-582a2f2f457a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:25.834692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:25.836581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:25.838404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:25.838404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:25.838404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:25.838404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:25.838404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:25.838404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:25.838404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:25.838404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:25.838404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 23 01:07:25.838404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:25.846688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 432/843] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:07:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:25.849250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e65dce-bae0-40a6-8b4a-582a2f2f457a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:25.874091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb813ed1-bf5a-41ef-b6c1-af9cfda1baa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:25.875944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb813ed1-bf5a-41ef-b6c1-af9cfda1baa9 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:25.876533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb813ed1-bf5a-41ef-b6c1-af9cfda1baa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:25.877124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb813ed1-bf5a-41ef-b6c1-af9cfda1baa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:25.878770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb813ed1-bf5a-41ef-b6c1-af9cfda1baa9 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=1c4da5b2130d4fa990adb72f0f6ec2fe) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:25.879439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb813ed1-bf5a-41ef-b6c1-af9cfda1baa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:25.882347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb813ed1-bf5a-41ef-b6c1-af9cfda1baa9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:25.899878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 412/844] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:07:25 2022] GET /identity/v3/role_assignments?user.id=1c4da5b2130d4fa990adb72f0f6ec2fe&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:25.987872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e65dce-bae0-40a6-8b4a-582a2f2f457a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:25.991687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:26.036204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41995d-643d-441c-93d3-2baf4cacd9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:26.037501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41995d-643d-441c-93d3-2baf4cacd9dc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:26.037976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41995d-643d-441c-93d3-2baf4cacd9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:26.038425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41995d-643d-441c-93d3-2baf4cacd9dc None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:26.039456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41995d-643d-441c-93d3-2baf4cacd9dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:26.040073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41995d-643d-441c-93d3-2baf4cacd9dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:26.042365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41995d-643d-441c-93d3-2baf4cacd9dc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:26.069317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 433/845] 10.222.0.64 () {56 vars in 1135 bytes} [Fri Sep 23 01:07:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:26.077224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb813ed1-bf5a-41ef-b6c1-af9cfda1baa9 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:26.166209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:26.263454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb813ed1-bf5a-41ef-b6c1-af9cfda1baa9 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:26.307397 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956a473-cfa9-4de8-aed5-db0beebb32eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:26.308366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956a473-cfa9-4de8-aed5-db0beebb32eb None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:26.308816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956a473-cfa9-4de8-aed5-db0beebb32eb None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:26.309241 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956a473-cfa9-4de8-aed5-db0beebb32eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:26.407405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956a473-cfa9-4de8-aed5-db0beebb32eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:26.409340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956a473-cfa9-4de8-aed5-db0beebb32eb None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:26.486335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:26.608441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 413/846] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:07:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:07:26.614704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41995d-643d-441c-93d3-2baf4cacd9dc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:26.805743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41995d-643d-441c-93d3-2baf4cacd9dc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:26.880517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172958dd-95c6-4157-81b5-6dbb3b38c688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:26.881982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172958dd-95c6-4157-81b5-6dbb3b38c688 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:26.882748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172958dd-95c6-4157-81b5-6dbb3b38c688 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:26.883471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172958dd-95c6-4157-81b5-6dbb3b38c688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:26.910500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172958dd-95c6-4157-81b5-6dbb3b38c688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:26.913433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172958dd-95c6-4157-81b5-6dbb3b38c688 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-172958dd-95c6-4157-81b5-6dbb3b38c688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:26.946104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:26.947231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:26.948201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:26.948201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:26.948201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:26.948865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 434/847] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:26.956201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956a473-cfa9-4de8-aed5-db0beebb32eb None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:27.104300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956a473-cfa9-4de8-aed5-db0beebb32eb None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:27.169025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf91a7-7925-48ac-b4b2-d2bd2a1e9fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:27.169949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf91a7-7925-48ac-b4b2-d2bd2a1e9fab None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:27.170397 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf91a7-7925-48ac-b4b2-d2bd2a1e9fab None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:27.170852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf91a7-7925-48ac-b4b2-d2bd2a1e9fab None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:27.172686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf91a7-7925-48ac-b4b2-d2bd2a1e9fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:27.173246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf91a7-7925-48ac-b4b2-d2bd2a1e9fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:27.175784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf91a7-7925-48ac-b4b2-d2bd2a1e9fab None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:27.192834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:27.199908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 414/848] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:27.206723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172958dd-95c6-4157-81b5-6dbb3b38c688 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:27.231939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:27.394037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172958dd-95c6-4157-81b5-6dbb3b38c688 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:27.462822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca0eda-b443-45aa-9827-4fa45d19be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:27.463921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0eda-b443-45aa-9827-4fa45d19be60 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:27.464413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0eda-b443-45aa-9827-4fa45d19be60 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:27.464908 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca0eda-b443-45aa-9827-4fa45d19be60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:27.478202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca0eda-b443-45aa-9827-4fa45d19be60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:27.481088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca0eda-b443-45aa-9827-4fa45d19be60 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-60ca0eda-b443-45aa-9827-4fa45d19be60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:27.500406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:27.501673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:27.502632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:27.502632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:27.502632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:27.502632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:27.502632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:27.502632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:27.502632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:27.502632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:27.502632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 435/849] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:27.510881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf91a7-7925-48ac-b4b2-d2bd2a1e9fab None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:27.687133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf91a7-7925-48ac-b4b2-d2bd2a1e9fab None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:27.758519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85a9a8-1f47-4b2c-bde1-bbb6915fd6f3 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:27.759033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85a9a8-1f47-4b2c-bde1-bbb6915fd6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:27.759173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85a9a8-1f47-4b2c-bde1-bbb6915fd6f3 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:27.761952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82290699-55e8-4d88-a255-0244e9a744b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:27.767451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290699-55e8-4d88-a255-0244e9a744b2 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:27.768413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290699-55e8-4d88-a255-0244e9a744b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:27.770201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290699-55e8-4d88-a255-0244e9a744b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:27.770955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 436/850] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:27.774956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cf505-fefc-4b70-962d-4aae66183bdd None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:27.774956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cf505-fefc-4b70-962d-4aae66183bdd None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:27.774956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cf505-fefc-4b70-962d-4aae66183bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:27.775766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290699-55e8-4d88-a255-0244e9a744b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:27.776634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290699-55e8-4d88-a255-0244e9a744b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:27.780747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290699-55e8-4d88-a255-0244e9a744b2 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:27.806504 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 415/851] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:27.813843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-0f5cf505-fefc-4b70-962d-4aae66183bdd None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:27.819021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82290699-55e8-4d88-a255-0244e9a744b2 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:27.822985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0f5cf505-fefc-4b70-962d-4aae66183bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:27.965948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 437/852] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:28.068062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:28.194139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82290699-55e8-4d88-a255-0244e9a744b2 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:28.277517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820e6c8-5505-4835-a8b0-5c5638d3d7bc None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:28.277850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820e6c8-5505-4835-a8b0-5c5638d3d7bc None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:28.278089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f820e6c8-5505-4835-a8b0-5c5638d3d7bc None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:28.279813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdafbb8-1118-4164-bdc6-5185f5811aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:28.279961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 438/853] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:28.281128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdafbb8-1118-4164-bdc6-5185f5811aee None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:28.281354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdafbb8-1118-4164-bdc6-5185f5811aee None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:28.281610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdafbb8-1118-4164-bdc6-5185f5811aee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:28.282408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdafbb8-1118-4164-bdc6-5185f5811aee None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=c5a9f3eca3e34fffa7b218064e831d7b) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:28.282795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdafbb8-1118-4164-bdc6-5185f5811aee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:28.284364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdafbb8-1118-4164-bdc6-5185f5811aee None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:28.292786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16d49f-61eb-450e-91a5-0f6384ff82e4 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:28.293104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16d49f-61eb-450e-91a5-0f6384ff82e4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:28.293408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16d49f-61eb-450e-91a5-0f6384ff82e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:28.297043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 416/854] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:07:27 2022] GET /identity/v3/role_assignments?user.id=c5a9f3eca3e34fffa7b218064e831d7b&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 608 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:28.302764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:28.334686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5b16d49f-61eb-450e-91a5-0f6384ff82e4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:28.340576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5b16d49f-61eb-450e-91a5-0f6384ff82e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:28.464751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdafbb8-1118-4164-bdc6-5185f5811aee None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:28.542982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 439/855] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:28.691205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdafbb8-1118-4164-bdc6-5185f5811aee None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:28.723079 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:28.736145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5be00-8a77-4673-bee3-a4e6e5956a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:28.745444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5be00-8a77-4673-bee3-a4e6e5956a2c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:28.745444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5be00-8a77-4673-bee3-a4e6e5956a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:28.745444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5be00-8a77-4673-bee3-a4e6e5956a2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:28.755636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5be00-8a77-4673-bee3-a4e6e5956a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-39a5be00-8a77-4673-bee3-a4e6e5956a2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:28.765223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:28.766540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:07:28.767386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:28.768117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 417/856] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:07:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:28.777818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16d49f-61eb-450e-91a5-0f6384ff82e4 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:28.952484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b16d49f-61eb-450e-91a5-0f6384ff82e4 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:28.968693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5be00-8a77-4673-bee3-a4e6e5956a2c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:29.021513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7bc720-8af3-48f5-b72e-f5cb11feba20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:29.022850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7bc720-8af3-48f5-b72e-f5cb11feba20 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:29.023450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7bc720-8af3-48f5-b72e-f5cb11feba20 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:29.024004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7bc720-8af3-48f5-b72e-f5cb11feba20 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:29.025681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7bc720-8af3-48f5-b72e-f5cb11feba20 None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:29.026181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7bc720-8af3-48f5-b72e-f5cb11feba20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:29.030558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7bc720-8af3-48f5-b72e-f5cb11feba20 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:29.035156 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:29.056688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 440/857] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:07:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:29.067028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7bc720-8af3-48f5-b72e-f5cb11feba20 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:29.206576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5be00-8a77-4673-bee3-a4e6e5956a2c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:29.281622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89828e3b-a7ec-4846-a154-e6601128e198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:29.282796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89828e3b-a7ec-4846-a154-e6601128e198 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:29.283458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89828e3b-a7ec-4846-a154-e6601128e198 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:29.283986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89828e3b-a7ec-4846-a154-e6601128e198 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:29.285552 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89828e3b-a7ec-4846-a154-e6601128e198 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:29.288308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89828e3b-a7ec-4846-a154-e6601128e198 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:29.290152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7bc720-8af3-48f5-b72e-f5cb11feba20 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:29.360926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b526cb1-bce5-4b57-a542-b344d2024c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:29.361770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b526cb1-bce5-4b57-a542-b344d2024c43 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:29.362058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b526cb1-bce5-4b57-a542-b344d2024c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:29.362337 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b526cb1-bce5-4b57-a542-b344d2024c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:29.386433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b526cb1-bce5-4b57-a542-b344d2024c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:29.389460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b526cb1-bce5-4b57-a542-b344d2024c43 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:29.413789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 418/858] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:07:28 2022] POST /identity/v3/services => generated 251 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:07:29.416347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448058ac-958f-4733-b39e-a1c4cc93c5ec None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:29.417056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448058ac-958f-4733-b39e-a1c4cc93c5ec None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:29.417731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448058ac-958f-4733-b39e-a1c4cc93c5ec None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-6b526cb1-bce5-4b57-a542-b344d2024c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:29.419528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:29.421819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:29.423729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:29.423729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:29.423729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:29.423729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 441/859] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:29.432786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 419/860] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:29 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:29.433040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b526cb1-bce5-4b57-a542-b344d2024c43 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:29.434614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f554f7-f22e-4fe8-944c-acdad3732442 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:29.435106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f554f7-f22e-4fe8-944c-acdad3732442 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:29.435489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f554f7-f22e-4fe8-944c-acdad3732442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:29.465680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-44f554f7-f22e-4fe8-944c-acdad3732442 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:29.467660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-44f554f7-f22e-4fe8-944c-acdad3732442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:29.548073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:29.550693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:29.607606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 420/861] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:29 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:29.647449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b526cb1-bce5-4b57-a542-b344d2024c43 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:29.698163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd918be-874a-485b-a49f-24da91b8a4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:29.699186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd918be-874a-485b-a49f-24da91b8a4f1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:29.699735 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd918be-874a-485b-a49f-24da91b8a4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:29.700119 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd918be-874a-485b-a49f-24da91b8a4f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:29.700842 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd918be-874a-485b-a49f-24da91b8a4f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:29.701150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd918be-874a-485b-a49f-24da91b8a4f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:29.702611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd918be-874a-485b-a49f-24da91b8a4f1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:29.718773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 442/862] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:29.729385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f554f7-f22e-4fe8-944c-acdad3732442 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:29.913772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f554f7-f22e-4fe8-944c-acdad3732442 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:29.969872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd17f33-4b18-4221-9bdc-216dc775befa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:29.970434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd17f33-4b18-4221-9bdc-216dc775befa None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:29.970717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd17f33-4b18-4221-9bdc-216dc775befa None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:29.971062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd17f33-4b18-4221-9bdc-216dc775befa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:29.980954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd17f33-4b18-4221-9bdc-216dc775befa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:29.982498 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd17f33-4b18-4221-9bdc-216dc775befa None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-4dd17f33-4b18-4221-9bdc-216dc775befa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:29.997364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:29.999148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:30.001692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 421/863] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Sep 23 01:07:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:30.005109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd918be-874a-485b-a49f-24da91b8a4f1 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:30.050434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd17f33-4b18-4221-9bdc-216dc775befa None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:30.112676 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:30.183481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd918be-874a-485b-a49f-24da91b8a4f1 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:30.200515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:30.234195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90300809-d574-43f3-813f-9758894ed00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:30.234849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90300809-d574-43f3-813f-9758894ed00e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:30.235133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90300809-d574-43f3-813f-9758894ed00e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:30.235412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90300809-d574-43f3-813f-9758894ed00e None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:30.236598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90300809-d574-43f3-813f-9758894ed00e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:30.236931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90300809-d574-43f3-813f-9758894ed00e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:30.238534 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90300809-d574-43f3-813f-9758894ed00e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:30.247886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd17f33-4b18-4221-9bdc-216dc775befa None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:30.267859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 443/864] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Sep 23 01:07:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:30.269814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90300809-d574-43f3-813f-9758894ed00e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:30.345646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053be97-87dc-4acd-88d1-989991223e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:30.346964 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053be97-87dc-4acd-88d1-989991223e18 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:30.348440 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053be97-87dc-4acd-88d1-989991223e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:30.349058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053be97-87dc-4acd-88d1-989991223e18 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:30.350720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053be97-87dc-4acd-88d1-989991223e18 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:30.352894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053be97-87dc-4acd-88d1-989991223e18 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-f053be97-87dc-4acd-88d1-989991223e18 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:30.374343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:30.376728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:30.378476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 422/865] 10.222.0.64 () {56 vars in 1132 bytes} [Fri Sep 23 01:07:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:30.386510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053be97-87dc-4acd-88d1-989991223e18 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:30.459683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90300809-d574-43f3-813f-9758894ed00e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:30.527489 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fa6cd-8c3b-4cf6-9cd5-25375a6dbc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:30.529282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fa6cd-8c3b-4cf6-9cd5-25375a6dbc99 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:30.530138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fa6cd-8c3b-4cf6-9cd5-25375a6dbc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:30.530924 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fa6cd-8c3b-4cf6-9cd5-25375a6dbc99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:30.580072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:30.591024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053be97-87dc-4acd-88d1-989991223e18 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:30.606578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:30.660947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817171e7-3d9a-4073-8252-d560a57b464d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:30.662058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817171e7-3d9a-4073-8252-d560a57b464d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:30.662771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817171e7-3d9a-4073-8252-d560a57b464d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:30.663266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817171e7-3d9a-4073-8252-d560a57b464d None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:30.664712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817171e7-3d9a-4073-8252-d560a57b464d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:30.665273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817171e7-3d9a-4073-8252-d560a57b464d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:30.668254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817171e7-3d9a-4073-8252-d560a57b464d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:30.685190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 423/866] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:30.687151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817171e7-3d9a-4073-8252-d560a57b464d None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:30.697056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fa6cd-8c3b-4cf6-9cd5-25375a6dbc99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:30.700592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fa6cd-8c3b-4cf6-9cd5-25375a6dbc99 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:30.896282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817171e7-3d9a-4073-8252-d560a57b464d None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:30.913751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 444/867] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:07:30 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 01:07:30.920817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fa6cd-8c3b-4cf6-9cd5-25375a6dbc99 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:30.949021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718cfac-8c4b-4fa1-aea9-815ffa7ac9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:30.949643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718cfac-8c4b-4fa1-aea9-815ffa7ac9ae None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:30.949943 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718cfac-8c4b-4fa1-aea9-815ffa7ac9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:30.950190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718cfac-8c4b-4fa1-aea9-815ffa7ac9ae None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:30.950839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718cfac-8c4b-4fa1-aea9-815ffa7ac9ae None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:30.951146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718cfac-8c4b-4fa1-aea9-815ffa7ac9ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:30.953312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718cfac-8c4b-4fa1-aea9-815ffa7ac9ae None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:30.959594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 424/868] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:31.054737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:31.126006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fa6cd-8c3b-4cf6-9cd5-25375a6dbc99 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:31.176334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6475e8-63d6-43db-abad-3440c30ac10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:31.177105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6475e8-63d6-43db-abad-3440c30ac10f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:31.177395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6475e8-63d6-43db-abad-3440c30ac10f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:31.177675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6475e8-63d6-43db-abad-3440c30ac10f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:31.192621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6475e8-63d6-43db-abad-3440c30ac10f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:31.194458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6475e8-63d6-43db-abad-3440c30ac10f None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-4c6475e8-63d6-43db-abad-3440c30ac10f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:31.215542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:31.216626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:31.217826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:31.217826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:31.217826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:31.217826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 445/869] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:31.225321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718cfac-8c4b-4fa1-aea9-815ffa7ac9ae None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:31.260515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:31.364805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718cfac-8c4b-4fa1-aea9-815ffa7ac9ae None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:31.410127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c578821-a69e-466d-be4a-14ef842e032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:31.411114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c578821-a69e-466d-be4a-14ef842e032e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:31.411749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c578821-a69e-466d-be4a-14ef842e032e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:31.412324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c578821-a69e-466d-be4a-14ef842e032e None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:31.413348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c578821-a69e-466d-be4a-14ef842e032e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:31.413891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c578821-a69e-466d-be4a-14ef842e032e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:31.415591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c578821-a69e-466d-be4a-14ef842e032e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:31.435958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 425/870] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:31.439365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6475e8-63d6-43db-abad-3440c30ac10f None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:31.634963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6475e8-63d6-43db-abad-3440c30ac10f None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:31.663878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:31.681275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc98ed-1b90-4893-bd34-22b796a2c621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:31.682020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc98ed-1b90-4893-bd34-22b796a2c621 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:31.696160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc98ed-1b90-4893-bd34-22b796a2c621 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:31.696160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc98ed-1b90-4893-bd34-22b796a2c621 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:31.696160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc98ed-1b90-4893-bd34-22b796a2c621 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:31.696160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc98ed-1b90-4893-bd34-22b796a2c621 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-48bc98ed-1b90-4893-bd34-22b796a2c621 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:31.709088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:31.711249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:31.713394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:31.713394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:31.713394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:31.713394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:31.713394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:31.713394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:31.713394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:31.713394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:31.713394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 446/871] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:31.727887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c578821-a69e-466d-be4a-14ef842e032e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:31.869722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696003c1-8648-4584-bd06-6d899b6c0888 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:31.870188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696003c1-8648-4584-bd06-6d899b6c0888 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:31.870684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696003c1-8648-4584-bd06-6d899b6c0888 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:31.875116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 447/872] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:07:31.883216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da5b80-6893-43f0-b940-889c6d822cbf None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:31.883739 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da5b80-6893-43f0-b940-889c6d822cbf None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:31.884212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da5b80-6893-43f0-b940-889c6d822cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:31.907974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:31.923725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-04da5b80-6893-43f0-b940-889c6d822cbf None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:31.923725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-04da5b80-6893-43f0-b940-889c6d822cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:31.939811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c578821-a69e-466d-be4a-14ef842e032e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:32.012619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2889-a18f-4938-931a-c1278c140b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:32.014097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2889-a18f-4938-931a-c1278c140b50 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:32.014850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2889-a18f-4938-931a-c1278c140b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:32.016397 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b2889-a18f-4938-931a-c1278c140b50 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:32.017766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b2889-a18f-4938-931a-c1278c140b50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:32.018297 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b2889-a18f-4938-931a-c1278c140b50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:32.021424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b2889-a18f-4938-931a-c1278c140b50 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:32.055766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 426/873] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:32.059926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2889-a18f-4938-931a-c1278c140b50 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:32.076686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 448/874] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:32.272957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b2889-a18f-4938-931a-c1278c140b50 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:32.344418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd4410-244c-4087-a401-01b61ef44dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:32.345938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd4410-244c-4087-a401-01b61ef44dbe None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:32.346737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd4410-244c-4087-a401-01b61ef44dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:32.347487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd4410-244c-4087-a401-01b61ef44dbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:32.349167 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd4410-244c-4087-a401-01b61ef44dbe None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=fa998fcd51b3404aab5412323f86908f) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:32.350002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd4410-244c-4087-a401-01b61ef44dbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:32.353076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd4410-244c-4087-a401-01b61ef44dbe None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:32.366615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 427/875] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:07:32 2022] GET /identity/v3/role_assignments?user.id=fa998fcd51b3404aab5412323f86908f&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:32.607466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da5b80-6893-43f0-b940-889c6d822cbf None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:32.763766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da5b80-6893-43f0-b940-889c6d822cbf None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:32.770313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f576be-779d-4ea9-90af-a4e2f728b081 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:32.770669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f576be-779d-4ea9-90af-a4e2f728b081 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:32.770887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f576be-779d-4ea9-90af-a4e2f728b081 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:32.772811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 428/876] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:07:32.778714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45275cf8-97cc-432f-a8b7-5486f84041df None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:32.778966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:32.779100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45275cf8-97cc-432f-a8b7-5486f84041df None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:32.779411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45275cf8-97cc-432f-a8b7-5486f84041df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:32.801036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:32.809737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-45275cf8-97cc-432f-a8b7-5486f84041df None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:32.811055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-45275cf8-97cc-432f-a8b7-5486f84041df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:32.832167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb7763-aff5-4526-99c8-caba15c67544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:32.833298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb7763-aff5-4526-99c8-caba15c67544 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:32.833797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb7763-aff5-4526-99c8-caba15c67544 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:32.834285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb7763-aff5-4526-99c8-caba15c67544 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:32.835389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb7763-aff5-4526-99c8-caba15c67544 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:32.838022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb7763-aff5-4526-99c8-caba15c67544 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-2fdb7763-aff5-4526-99c8-caba15c67544 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:32.857729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:32.858866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:32.860394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:32.861214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 449/877] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:32.870438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb7763-aff5-4526-99c8-caba15c67544 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:32.922644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 429/878] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:33.084955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb7763-aff5-4526-99c8-caba15c67544 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:33.153207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb560f-e10c-4f54-8401-6a882696d923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:33.154694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb560f-e10c-4f54-8401-6a882696d923 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:33.155430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb560f-e10c-4f54-8401-6a882696d923 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:33.156314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb560f-e10c-4f54-8401-6a882696d923 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:33.157779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb560f-e10c-4f54-8401-6a882696d923 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:33.158608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb560f-e10c-4f54-8401-6a882696d923 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:33.161521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb560f-e10c-4f54-8401-6a882696d923 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:33.172183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 450/879] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Sep 23 01:07:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:33.179607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45275cf8-97cc-432f-a8b7-5486f84041df None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:33.287400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb560f-e10c-4f54-8401-6a882696d923 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:33.369825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45275cf8-97cc-432f-a8b7-5486f84041df None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:33.379033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:33.462773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994745f-982e-4d2b-a6f9-8eb0a94c4388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:33.463826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994745f-982e-4d2b-a6f9-8eb0a94c4388 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:33.464309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994745f-982e-4d2b-a6f9-8eb0a94c4388 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:33.464772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994745f-982e-4d2b-a6f9-8eb0a94c4388 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:33.465943 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994745f-982e-4d2b-a6f9-8eb0a94c4388 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:33.466481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994745f-982e-4d2b-a6f9-8eb0a94c4388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:33.470107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994745f-982e-4d2b-a6f9-8eb0a94c4388 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:33.475088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:33.482479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 430/880] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Sep 23 01:07:33 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:33.531352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb560f-e10c-4f54-8401-6a882696d923 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:33.604347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abd057-c334-4512-a2a3-8229ff578842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:33.605547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abd057-c334-4512-a2a3-8229ff578842 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:33.606030 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abd057-c334-4512-a2a3-8229ff578842 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:33.606509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abd057-c334-4512-a2a3-8229ff578842 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:33.607710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abd057-c334-4512-a2a3-8229ff578842 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:33.610627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abd057-c334-4512-a2a3-8229ff578842 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-a9abd057-c334-4512-a2a3-8229ff578842 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:33.630334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:33.632358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:33.634044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 451/881] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:07:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:33.643601 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994745f-982e-4d2b-a6f9-8eb0a94c4388 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:33.861401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994745f-982e-4d2b-a6f9-8eb0a94c4388 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:33.931762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19d0e-ebe0-4bc0-b158-dea50d5ec46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:33.932843 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19d0e-ebe0-4bc0-b158-dea50d5ec46a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:33.933343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19d0e-ebe0-4bc0-b158-dea50d5ec46a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:33.933842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19d0e-ebe0-4bc0-b158-dea50d5ec46a None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:33.935032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b19d0e-ebe0-4bc0-b158-dea50d5ec46a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:33.935619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b19d0e-ebe0-4bc0-b158-dea50d5ec46a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:33.938401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b19d0e-ebe0-4bc0-b158-dea50d5ec46a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:33.947947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 431/882] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:07:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:33.957276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abd057-c334-4512-a2a3-8229ff578842 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:33.977187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913db50-351d-4508-a8b5-76145e47aa52 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:33.977695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913db50-351d-4508-a8b5-76145e47aa52 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:33.978201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d913db50-351d-4508-a8b5-76145e47aa52 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:33.981300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 432/883] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:07:33.990700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf40ec0-f415-4e4a-a6f1-798f09fba35a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:33.991211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf40ec0-f415-4e4a-a6f1-798f09fba35a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:33.991734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf40ec0-f415-4e4a-a6f1-798f09fba35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:34.024194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-eaf40ec0-f415-4e4a-a6f1-798f09fba35a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:34.026429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-eaf40ec0-f415-4e4a-a6f1-798f09fba35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:34.148207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:34.156715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:34.162812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abd057-c334-4512-a2a3-8229ff578842 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:34.167064 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 433/884] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:33 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:34.226974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c918376-aff4-4381-b823-f91def9a013d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:34.228035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c918376-aff4-4381-b823-f91def9a013d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:34.228565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c918376-aff4-4381-b823-f91def9a013d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:34.229104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c918376-aff4-4381-b823-f91def9a013d None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:34.230414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c918376-aff4-4381-b823-f91def9a013d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:34.231042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c918376-aff4-4381-b823-f91def9a013d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:34.234646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c918376-aff4-4381-b823-f91def9a013d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:34.244815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 452/885] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:07:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:34.254526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf40ec0-f415-4e4a-a6f1-798f09fba35a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:34.463489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf40ec0-f415-4e4a-a6f1-798f09fba35a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:34.528696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f21915-ec4b-4458-b8e8-ef9670709322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:34.529658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f21915-ec4b-4458-b8e8-ef9670709322 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:34.530159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f21915-ec4b-4458-b8e8-ef9670709322 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:34.530708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f21915-ec4b-4458-b8e8-ef9670709322 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:34.532142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f21915-ec4b-4458-b8e8-ef9670709322 None admin] RBAC: Adding query filter params (interface=public, service_id=d3ed4ca515d448c1a23e99326c0816ce, region_id=RegionOne) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:34.532704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f21915-ec4b-4458-b8e8-ef9670709322 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:34.535495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f21915-ec4b-4458-b8e8-ef9670709322 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:34.546853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 434/886] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Sep 23 01:07:34 2022] GET /identity/v3/endpoints?service_id=d3ed4ca515d448c1a23e99326c0816ce&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:34.555956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c918376-aff4-4381-b823-f91def9a013d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:34.681319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f21915-ec4b-4458-b8e8-ef9670709322 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:34.721426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c918376-aff4-4381-b823-f91def9a013d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:34.805399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e98e8f-50bf-4b6b-96f7-1bbbbd5b3e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:34.806040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e98e8f-50bf-4b6b-96f7-1bbbbd5b3e85 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:34.806320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e98e8f-50bf-4b6b-96f7-1bbbbd5b3e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:34.806600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e98e8f-50bf-4b6b-96f7-1bbbbd5b3e85 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:34.807242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e98e8f-50bf-4b6b-96f7-1bbbbd5b3e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:34.812299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e98e8f-50bf-4b6b-96f7-1bbbbd5b3e85 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:34.812709 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:34.820636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 453/887] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Sep 23 01:07:34 2022] GET /identity/v3/services => generated 1011 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:07:34.877820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f21915-ec4b-4458-b8e8-ef9670709322 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:34.950546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710968e-b432-4931-ae44-e81090067960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:34.951675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710968e-b432-4931-ae44-e81090067960 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:34.952159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710968e-b432-4931-ae44-e81090067960 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:34.952715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710968e-b432-4931-ae44-e81090067960 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:34.954026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710968e-b432-4931-ae44-e81090067960 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:34.956895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710968e-b432-4931-ae44-e81090067960 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:35.057871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 435/888] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:07:34 2022] POST /identity/v3/services => generated 269 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:07:36.102222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408c424-ada6-4a59-99b7-6b84cc8b2de9 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:36.102962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408c424-ada6-4a59-99b7-6b84cc8b2de9 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:36.102962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0408c424-ada6-4a59-99b7-6b84cc8b2de9 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:36.104624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 454/889] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:36.111611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8551919-75a8-4af8-ab8c-73ced57c771c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:36.112495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8551919-75a8-4af8-ab8c-73ced57c771c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:36.112495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8551919-75a8-4af8-ab8c-73ced57c771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:36.141593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a8551919-75a8-4af8-ab8c-73ced57c771c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:36.143775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a8551919-75a8-4af8-ab8c-73ced57c771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:36.287689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 436/890] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:36.763678 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408c424-ada6-4a59-99b7-6b84cc8b2de9 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:36.823123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:36.980949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee275a-8a75-4d89-928d-db3737026165 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:36.981491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee275a-8a75-4d89-928d-db3737026165 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:36.982124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee275a-8a75-4d89-928d-db3737026165 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:36.991386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 437/891] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:36.994669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0408c424-ada6-4a59-99b7-6b84cc8b2de9 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:36.995449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c4a4-5b62-4241-9caa-67dcfd87bb8d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:36.995967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c4a4-5b62-4241-9caa-67dcfd87bb8d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:36.996467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c4a4-5b62-4241-9caa-67dcfd87bb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:37.029131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b1f8c4a4-5b62-4241-9caa-67dcfd87bb8d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:37.031454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b1f8c4a4-5b62-4241-9caa-67dcfd87bb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:37.070067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ca25c-4300-4705-85b7-1654a2337c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:37.071641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ca25c-4300-4705-85b7-1654a2337c6f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:37.072442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ca25c-4300-4705-85b7-1654a2337c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:37.073191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ca25c-4300-4705-85b7-1654a2337c6f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:37.095828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ca25c-4300-4705-85b7-1654a2337c6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-bf6ca25c-4300-4705-85b7-1654a2337c6f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:37.106046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:37.108466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:07:37.111089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:37.115513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 455/892] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Sep 23 01:07:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:37.125400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ca25c-4300-4705-85b7-1654a2337c6f None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:37.131345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:37.179588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 438/893] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:37.310480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:37.335204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ca25c-4300-4705-85b7-1654a2337c6f None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:37.388742 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cec592-5df0-4d4a-9852-b2d201846309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:37.389399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cec592-5df0-4d4a-9852-b2d201846309 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:37.389689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cec592-5df0-4d4a-9852-b2d201846309 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:37.390578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cec592-5df0-4d4a-9852-b2d201846309 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:37.391292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cec592-5df0-4d4a-9852-b2d201846309 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:37.391645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cec592-5df0-4d4a-9852-b2d201846309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:37.393182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cec592-5df0-4d4a-9852-b2d201846309 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:37.407884 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 456/894] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Sep 23 01:07:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:37.411371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c4a4-5b62-4241-9caa-67dcfd87bb8d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:37.570640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c4a4-5b62-4241-9caa-67dcfd87bb8d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:37.619737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef92d6-f768-4190-b8e1-2631677b6912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:37.620512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef92d6-f768-4190-b8e1-2631677b6912 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:37.620821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef92d6-f768-4190-b8e1-2631677b6912 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:37.621097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef92d6-f768-4190-b8e1-2631677b6912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:37.637720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef92d6-f768-4190-b8e1-2631677b6912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:37.639255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef92d6-f768-4190-b8e1-2631677b6912 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-05ef92d6-f768-4190-b8e1-2631677b6912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:37.670697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:37.673121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:37.674908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:37.674908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:37.674908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:37.678850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 439/895] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:37.689121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cec592-5df0-4d4a-9852-b2d201846309 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:37.706181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef92d6-f768-4190-b8e1-2631677b6912 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:37.767203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:37.906657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cec592-5df0-4d4a-9852-b2d201846309 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:37.958024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef92d6-f768-4190-b8e1-2631677b6912 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:37.991011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ab482-02a9-42a5-8575-53e5312405e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:37.992417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ab482-02a9-42a5-8575-53e5312405e9 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:37.992753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ab482-02a9-42a5-8575-53e5312405e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:37.993237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ab482-02a9-42a5-8575-53e5312405e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:37.994220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ab482-02a9-42a5-8575-53e5312405e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:37.994757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ab482-02a9-42a5-8575-53e5312405e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:37.997452 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ab482-02a9-42a5-8575-53e5312405e9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:38.025324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 457/896] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:38.033998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ab482-02a9-42a5-8575-53e5312405e9 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:38.042365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400edb-3d46-4ae9-b218-944a10ee1d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:38.043434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400edb-3d46-4ae9-b218-944a10ee1d73 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:38.043981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400edb-3d46-4ae9-b218-944a10ee1d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:38.044285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78400edb-3d46-4ae9-b218-944a10ee1d73 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:38.045480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78400edb-3d46-4ae9-b218-944a10ee1d73 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:38.049158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78400edb-3d46-4ae9-b218-944a10ee1d73 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:38.055002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:38.216431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ab482-02a9-42a5-8575-53e5312405e9 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:38.251736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 440/897] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:07:37 2022] POST /identity/v3/services => generated 241 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:07:38.305024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88ca0d-ba82-4b38-9bfe-8ed6a393481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:38.306307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88ca0d-ba82-4b38-9bfe-8ed6a393481b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:38.306851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88ca0d-ba82-4b38-9bfe-8ed6a393481b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:38.307710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88ca0d-ba82-4b38-9bfe-8ed6a393481b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:38.324169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88ca0d-ba82-4b38-9bfe-8ed6a393481b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:38.327792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88ca0d-ba82-4b38-9bfe-8ed6a393481b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-0a88ca0d-ba82-4b38-9bfe-8ed6a393481b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:38.352371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:38.354179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:38.357216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 458/898] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Sep 23 01:07:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:38.364533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400edb-3d46-4ae9-b218-944a10ee1d73 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:38.565302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:38.603895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78400edb-3d46-4ae9-b218-944a10ee1d73 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:38.649691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52577503-59d2-44e4-9388-7e094843abae None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:38.650140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52577503-59d2-44e4-9388-7e094843abae None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:38.650584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52577503-59d2-44e4-9388-7e094843abae None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:38.653617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 459/899] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:07:38.663234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c013a8-2bc6-4851-acbb-e19e3d238c25 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:38.663758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c013a8-2bc6-4851-acbb-e19e3d238c25 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:38.664215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c013a8-2bc6-4851-acbb-e19e3d238c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:38.678985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb41e78-c25c-4456-bb9b-ff4721b6e209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:38.680717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb41e78-c25c-4456-bb9b-ff4721b6e209 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:38.681180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb41e78-c25c-4456-bb9b-ff4721b6e209 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:38.681641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb41e78-c25c-4456-bb9b-ff4721b6e209 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:38.683059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb41e78-c25c-4456-bb9b-ff4721b6e209 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:38.683602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb41e78-c25c-4456-bb9b-ff4721b6e209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:38.686371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb41e78-c25c-4456-bb9b-ff4721b6e209 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:38.700708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-09c013a8-2bc6-4851-acbb-e19e3d238c25 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:38.703488 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-09c013a8-2bc6-4851-acbb-e19e3d238c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:38.721354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 441/900] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Sep 23 01:07:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:38.729441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb41e78-c25c-4456-bb9b-ff4721b6e209 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:38.828814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:38.844231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 460/901] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:38.847609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4d921-a7bd-487c-adf8-55a84037f6b7 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:38.849376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4d921-a7bd-487c-adf8-55a84037f6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:38.850679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4d921-a7bd-487c-adf8-55a84037f6b7 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:38.854848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 461/902] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:38.864253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d66375-e5d7-4974-abb0-0df371f33fc9 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:38.864838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d66375-e5d7-4974-abb0-0df371f33fc9 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:38.865354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d66375-e5d7-4974-abb0-0df371f33fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:38.899479 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-93d66375-e5d7-4974-abb0-0df371f33fc9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:38.903014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-93d66375-e5d7-4974-abb0-0df371f33fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:38.963976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb41e78-c25c-4456-bb9b-ff4721b6e209 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:39.033995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 462/903] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:39.036346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c92854-797b-4e9f-868a-34ab37040b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:39.037706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92854-797b-4e9f-868a-34ab37040b6c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:39.038442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92854-797b-4e9f-868a-34ab37040b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:39.039179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c92854-797b-4e9f-868a-34ab37040b6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:39.069850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:39.180793 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c92854-797b-4e9f-868a-34ab37040b6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:39.184274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c92854-797b-4e9f-868a-34ab37040b6c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:39.209410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d66375-e5d7-4974-abb0-0df371f33fc9 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:39.358210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 442/904] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:07:38 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:07:39.364033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c92854-797b-4e9f-868a-34ab37040b6c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:39.398929 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d66375-e5d7-4974-abb0-0df371f33fc9 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:39.434950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:39.443432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:39.452732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e2edb-f630-4917-893e-790bb6316b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:39.453403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e2edb-f630-4917-893e-790bb6316b65 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:39.453684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e2edb-f630-4917-893e-790bb6316b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:39.453961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e2edb-f630-4917-893e-790bb6316b65 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:39.454569 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e2edb-f630-4917-893e-790bb6316b65 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:39.456536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e2edb-f630-4917-893e-790bb6316b65 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-db5e2edb-f630-4917-893e-790bb6316b65 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:39.470468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:39.473454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 01:07:39.475191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:39.478144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 463/905] 10.222.0.64 () {56 vars in 1132 bytes} [Fri Sep 23 01:07:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:39.479134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e2edb-f630-4917-893e-790bb6316b65 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:39.556138 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c92854-797b-4e9f-868a-34ab37040b6c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:39.625263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e2edb-f630-4917-893e-790bb6316b65 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:39.625729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bd2e4-f0e2-4b88-952f-b4b578f6ffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:39.626832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bd2e4-f0e2-4b88-952f-b4b578f6ffa5 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:39.627336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bd2e4-f0e2-4b88-952f-b4b578f6ffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:39.627877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bd2e4-f0e2-4b88-952f-b4b578f6ffa5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:39.628955 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bd2e4-f0e2-4b88-952f-b4b578f6ffa5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:39.631251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bd2e4-f0e2-4b88-952f-b4b578f6ffa5 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-063bd2e4-f0e2-4b88-952f-b4b578f6ffa5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:39.649948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:39.651122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:39.652162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 443/906] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:07:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:39.653957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bd2e4-f0e2-4b88-952f-b4b578f6ffa5 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:39.694205 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6aedf-18d3-46d2-aed0-45ab9b140212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:39.695223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6aedf-18d3-46d2-aed0-45ab9b140212 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:39.695759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6aedf-18d3-46d2-aed0-45ab9b140212 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:39.696231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6aedf-18d3-46d2-aed0-45ab9b140212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:39.716703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6aedf-18d3-46d2-aed0-45ab9b140212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:39.720067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6aedf-18d3-46d2-aed0-45ab9b140212 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-57d6aedf-18d3-46d2-aed0-45ab9b140212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:39.747429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:39.749414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:39.751208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:39.751208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:39.751208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:39.751208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 464/907] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:39.751208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6aedf-18d3-46d2-aed0-45ab9b140212 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:39.815591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:39.860409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bd2e4-f0e2-4b88-952f-b4b578f6ffa5 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:39.875003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:39.920373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc895b0b-562c-4e19-ab92-33749c291de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:39.926289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc895b0b-562c-4e19-ab92-33749c291de4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:39.926289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc895b0b-562c-4e19-ab92-33749c291de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:39.926289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc895b0b-562c-4e19-ab92-33749c291de4 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:39.928143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc895b0b-562c-4e19-ab92-33749c291de4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:39.928143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc895b0b-562c-4e19-ab92-33749c291de4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:39.930369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc895b0b-562c-4e19-ab92-33749c291de4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:39.947657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6aedf-18d3-46d2-aed0-45ab9b140212 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:39.947896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 444/908] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:39.947896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc895b0b-562c-4e19-ab92-33749c291de4 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:39.996741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3e83a-a87d-4420-a769-f8cc9751c2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:39.997885 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e83a-a87d-4420-a769-f8cc9751c2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:39.998357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e83a-a87d-4420-a769-f8cc9751c2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:39.998792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e83a-a87d-4420-a769-f8cc9751c2c3 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:40.000445 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e83a-a87d-4420-a769-f8cc9751c2c3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:40.000944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e83a-a87d-4420-a769-f8cc9751c2c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:40.003251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e83a-a87d-4420-a769-f8cc9751c2c3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:40.019720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 465/909] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:07:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:40.019720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3e83a-a87d-4420-a769-f8cc9751c2c3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:40.145654 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc895b0b-562c-4e19-ab92-33749c291de4 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:40.159992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3e83a-a87d-4420-a769-f8cc9751c2c3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:40.172638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:40.212728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be945ed0-40cd-49f3-bf53-b92b80c5d726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:40.213861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be945ed0-40cd-49f3-bf53-b92b80c5d726 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:40.214353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be945ed0-40cd-49f3-bf53-b92b80c5d726 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:40.214846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be945ed0-40cd-49f3-bf53-b92b80c5d726 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:40.216035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be945ed0-40cd-49f3-bf53-b92b80c5d726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:40.216580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be945ed0-40cd-49f3-bf53-b92b80c5d726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:40.219660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be945ed0-40cd-49f3-bf53-b92b80c5d726 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:40.223718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19694c2f-87c3-481b-98d6-f619c4cf1799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:40.225272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19694c2f-87c3-481b-98d6-f619c4cf1799 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:40.225974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19694c2f-87c3-481b-98d6-f619c4cf1799 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:40.226667 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19694c2f-87c3-481b-98d6-f619c4cf1799 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:40.228124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19694c2f-87c3-481b-98d6-f619c4cf1799 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:40.228906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19694c2f-87c3-481b-98d6-f619c4cf1799 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:40.231811 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19694c2f-87c3-481b-98d6-f619c4cf1799 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:40.241996 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 466/910] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:40.243238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19694c2f-87c3-481b-98d6-f619c4cf1799 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:40.243348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 445/911] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:40.251944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be945ed0-40cd-49f3-bf53-b92b80c5d726 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:40.313382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:40.457960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19694c2f-87c3-481b-98d6-f619c4cf1799 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:40.461147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be945ed0-40cd-49f3-bf53-b92b80c5d726 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:40.523079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b61836-598c-4ac3-8c44-a7d252b63573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:40.524018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b61836-598c-4ac3-8c44-a7d252b63573 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:40.524514 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b61836-598c-4ac3-8c44-a7d252b63573 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:40.525050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b61836-598c-4ac3-8c44-a7d252b63573 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:40.526190 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b61836-598c-4ac3-8c44-a7d252b63573 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:40.526822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b61836-598c-4ac3-8c44-a7d252b63573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:40.528131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd171-51c7-459f-b6ae-f6bdb5e44010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:40.528798 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd171-51c7-459f-b6ae-f6bdb5e44010 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:40.529083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd171-51c7-459f-b6ae-f6bdb5e44010 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:40.529234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b61836-598c-4ac3-8c44-a7d252b63573 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:40.529369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd171-51c7-459f-b6ae-f6bdb5e44010 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:40.530131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bd171-51c7-459f-b6ae-f6bdb5e44010 None admin] RBAC: Adding query filter params (interface=public, service_id=ec1a5e9dacaa41deaad4e99fe3839bae, region_id=RegionOne) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:40.530519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bd171-51c7-459f-b6ae-f6bdb5e44010 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:40.532009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bd171-51c7-459f-b6ae-f6bdb5e44010 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:40.543609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 467/912] 10.222.0.64 () {56 vars in 1114 bytes} [Fri Sep 23 01:07:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:40.543813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 446/913] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Sep 23 01:07:40 2022] GET /identity/v3/endpoints?service_id=ec1a5e9dacaa41deaad4e99fe3839bae&interface=public®ion_id=RegionOne => generated 194 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:40.543953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b61836-598c-4ac3-8c44-a7d252b63573 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:40.553322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd171-51c7-459f-b6ae-f6bdb5e44010 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:40.602068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:40.711905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b61836-598c-4ac3-8c44-a7d252b63573 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:40.756634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:40.762288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b3efa-4380-4423-8ef2-50dee1658d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:40.763377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b3efa-4380-4423-8ef2-50dee1658d13 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:40.763861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b3efa-4380-4423-8ef2-50dee1658d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:40.764261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b3efa-4380-4423-8ef2-50dee1658d13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:40.771007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd171-51c7-459f-b6ae-f6bdb5e44010 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:40.774093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b3efa-4380-4423-8ef2-50dee1658d13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:40.775678 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b3efa-4380-4423-8ef2-50dee1658d13 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-528b3efa-4380-4423-8ef2-50dee1658d13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:40.790878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:40.792261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:40.794072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:40.794072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:40.794072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:40.794072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:40.794072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:40.794072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:40.794072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:40.794072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:40.794072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 468/914] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:40.794072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b3efa-4380-4423-8ef2-50dee1658d13 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:40.858502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87022a-ac3a-4c9d-830d-b19b3376a796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:40.860305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87022a-ac3a-4c9d-830d-b19b3376a796 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:40.860913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87022a-ac3a-4c9d-830d-b19b3376a796 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:40.861433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e87022a-ac3a-4c9d-830d-b19b3376a796 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:40.863065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87022a-ac3a-4c9d-830d-b19b3376a796 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:40.865872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e87022a-ac3a-4c9d-830d-b19b3376a796 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:40.972930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:40.984361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b3efa-4380-4423-8ef2-50dee1658d13 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:40.992218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 447/915] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:07:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:07:40.994068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87022a-ac3a-4c9d-830d-b19b3376a796 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:41.074152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4f8e9-8732-4597-b5f5-dc5fe7f53b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:41.075274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4f8e9-8732-4597-b5f5-dc5fe7f53b70 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:41.075902 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4f8e9-8732-4597-b5f5-dc5fe7f53b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:41.076480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df4f8e9-8732-4597-b5f5-dc5fe7f53b70 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:41.078133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4f8e9-8732-4597-b5f5-dc5fe7f53b70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:41.080878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df4f8e9-8732-4597-b5f5-dc5fe7f53b70 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:41.095860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 469/916] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Sep 23 01:07:40 2022] GET /identity/v3/services => generated 1497 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:07:41.213620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e87022a-ac3a-4c9d-830d-b19b3376a796 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:41.286122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d29826-2c09-41d9-be69-4ab1c8d78293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:41.286852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d29826-2c09-41d9-be69-4ab1c8d78293 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:41.287313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d29826-2c09-41d9-be69-4ab1c8d78293 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:41.287830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d29826-2c09-41d9-be69-4ab1c8d78293 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:41.289158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d29826-2c09-41d9-be69-4ab1c8d78293 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:41.289693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d29826-2c09-41d9-be69-4ab1c8d78293 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:41.291953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d29826-2c09-41d9-be69-4ab1c8d78293 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:41.314833 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 448/917] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:41.326169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4f8e9-8732-4597-b5f5-dc5fe7f53b70 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:41.539287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df4f8e9-8732-4597-b5f5-dc5fe7f53b70 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:41.611009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c69daf5-75f2-43b0-93dc-04dc0995e584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:41.611982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69daf5-75f2-43b0-93dc-04dc0995e584 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:41.612434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69daf5-75f2-43b0-93dc-04dc0995e584 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:41.612876 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c69daf5-75f2-43b0-93dc-04dc0995e584 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:41.614218 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69daf5-75f2-43b0-93dc-04dc0995e584 None admin] RBAC: Adding query filter params (role.id=81d0f75541e3413d802620f47d89ed62, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=fa998fcd51b3404aab5412323f86908f) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:41.614723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69daf5-75f2-43b0-93dc-04dc0995e584 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:41.617520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c69daf5-75f2-43b0-93dc-04dc0995e584 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:41.629394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 470/918] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:07:41 2022] GET /identity/v3/role_assignments?user.id=fa998fcd51b3404aab5412323f86908f&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=81d0f75541e3413d802620f47d89ed62 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:41.651475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f77456-7481-40bb-9c09-3a670c8022b1 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:41.652183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f77456-7481-40bb-9c09-3a670c8022b1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:41.652183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f77456-7481-40bb-9c09-3a670c8022b1 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:41.653955 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 449/919] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:41.661274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b814438-e950-460f-8370-9fcf78b56b9b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:41.661558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b814438-e950-460f-8370-9fcf78b56b9b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:41.661836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b814438-e950-460f-8370-9fcf78b56b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:41.691794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3b814438-e950-460f-8370-9fcf78b56b9b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:41.694323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3b814438-e950-460f-8370-9fcf78b56b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:41.724963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:41.848341 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 471/920] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:42.225923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f77456-7481-40bb-9c09-3a670c8022b1 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:42.250431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:42.419141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f77456-7481-40bb-9c09-3a670c8022b1 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:42.486676 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41827b-b65a-4659-a19e-73f701630a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:42.487763 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41827b-b65a-4659-a19e-73f701630a78 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:42.488278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41827b-b65a-4659-a19e-73f701630a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:42.488787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41827b-b65a-4659-a19e-73f701630a78 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:42.490013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41827b-b65a-4659-a19e-73f701630a78 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:42.492850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa41827b-b65a-4659-a19e-73f701630a78 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-fa41827b-b65a-4659-a19e-73f701630a78 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:42.510484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:42.512454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:42.514085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 450/921] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:42.527973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b814438-e950-460f-8370-9fcf78b56b9b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:42.726308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b814438-e950-460f-8370-9fcf78b56b9b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:42.792048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db1760-a90b-4c02-8ca7-b319495567b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:42.792927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db1760-a90b-4c02-8ca7-b319495567b5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:42.793345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db1760-a90b-4c02-8ca7-b319495567b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:42.793761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db1760-a90b-4c02-8ca7-b319495567b5 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:42.794766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db1760-a90b-4c02-8ca7-b319495567b5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:42.795240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db1760-a90b-4c02-8ca7-b319495567b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:42.798271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db1760-a90b-4c02-8ca7-b319495567b5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:42.807855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 472/922] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Sep 23 01:07:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:42.818373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41827b-b65a-4659-a19e-73f701630a78 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:42.981825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:43.028610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa41827b-b65a-4659-a19e-73f701630a78 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:43.082303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115485f8-bff2-49d2-88ab-22da4f30b11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:43.082852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115485f8-bff2-49d2-88ab-22da4f30b11b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:43.083136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115485f8-bff2-49d2-88ab-22da4f30b11b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:43.083446 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115485f8-bff2-49d2-88ab-22da4f30b11b None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:43.084333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115485f8-bff2-49d2-88ab-22da4f30b11b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:43.084800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115485f8-bff2-49d2-88ab-22da4f30b11b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:43.086343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115485f8-bff2-49d2-88ab-22da4f30b11b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:43.093517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 451/923] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Sep 23 01:07:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:43.101270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db1760-a90b-4c02-8ca7-b319495567b5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:43.127943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:43.275246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db1760-a90b-4c02-8ca7-b319495567b5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:43.348015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a42e8-b846-48ca-8928-db7860769fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:43.348991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a42e8-b846-48ca-8928-db7860769fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:43.349495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a42e8-b846-48ca-8928-db7860769fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:43.349947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a42e8-b846-48ca-8928-db7860769fc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:43.351369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a42e8-b846-48ca-8928-db7860769fc9 None admin] RBAC: Adding query filter params (interface=public, service_id=f5150fe704124e19bebce65d1991c813, region_id=RegionOne) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:43.351950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a42e8-b846-48ca-8928-db7860769fc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:43.354666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a42e8-b846-48ca-8928-db7860769fc9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:43.366361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 473/924] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Sep 23 01:07:43 2022] GET /identity/v3/endpoints?service_id=f5150fe704124e19bebce65d1991c813&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:43.372886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115485f8-bff2-49d2-88ab-22da4f30b11b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:43.567006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115485f8-bff2-49d2-88ab-22da4f30b11b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:43.621572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a10d21c-dadb-4459-893a-2578b9c0dec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:43.622487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10d21c-dadb-4459-893a-2578b9c0dec4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:43.622995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10d21c-dadb-4459-893a-2578b9c0dec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:43.623531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a10d21c-dadb-4459-893a-2578b9c0dec4 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:43.625439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10d21c-dadb-4459-893a-2578b9c0dec4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:43.628329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a10d21c-dadb-4459-893a-2578b9c0dec4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:43.641534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 452/925] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Sep 23 01:07:43 2022] GET /identity/v3/services => generated 1497 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:07:44.258084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a5c3d-15eb-4e96-b28a-a38adfdfe98a None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:44.259804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a5c3d-15eb-4e96-b28a-a38adfdfe98a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:44.260595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a5c3d-15eb-4e96-b28a-a38adfdfe98a None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:44.264134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 474/926] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:44.274274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9027d-690b-4ee8-879a-e0e180d5a035 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:44.275735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9027d-690b-4ee8-879a-e0e180d5a035 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:44.276254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9027d-690b-4ee8-879a-e0e180d5a035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:44.287288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:44.306081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d7b9027d-690b-4ee8-879a-e0e180d5a035 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:44.308563 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d7b9027d-690b-4ee8-879a-e0e180d5a035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:44.434208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 453/927] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:44.733226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ad5a-a2e2-45f7-be5f-f33bfc723f76 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:44.734934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ad5a-a2e2-45f7-be5f-f33bfc723f76 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:44.735787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519ad5a-a2e2-45f7-be5f-f33bfc723f76 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:44.738853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 475/928] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:44.745545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52f170-7bd3-49c8-a8ac-089ac0c8319a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:44.745846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52f170-7bd3-49c8-a8ac-089ac0c8319a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:44.746276 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52f170-7bd3-49c8-a8ac-089ac0c8319a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:44.769678 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-5b52f170-7bd3-49c8-a8ac-089ac0c8319a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:44.771004 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5b52f170-7bd3-49c8-a8ac-089ac0c8319a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:44.794711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf06868-3e7d-45d9-a47c-ed4d9692b6c0 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:44.795153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf06868-3e7d-45d9-a47c-ed4d9692b6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:44.795622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf06868-3e7d-45d9-a47c-ed4d9692b6c0 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:44.799524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 476/929] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:44.806536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cad9c9-843c-4120-86b2-f3d9fbd55c9d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:44.806948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cad9c9-843c-4120-86b2-f3d9fbd55c9d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:44.807140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cad9c9-843c-4120-86b2-f3d9fbd55c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:44.847450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-e5cad9c9-843c-4120-86b2-f3d9fbd55c9d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:44.849186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e5cad9c9-843c-4120-86b2-f3d9fbd55c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:44.884517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 454/930] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:44.886025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52f170-7bd3-49c8-a8ac-089ac0c8319a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:44.971716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 477/931] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:45.004141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:45.051286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52f170-7bd3-49c8-a8ac-089ac0c8319a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:45.102273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:45.103447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:45.103993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:45.104498 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:45.105753 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:45.108392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:45.113643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: WARNING keystone.server.flask.common [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 23 01:07:45.174531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.sql.core [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 23 01:07:45.174531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 23 01:07:45.174531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [parameters: {'id': '4642b8e81dd64166abc35efc1d42bbd2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 23 01:07:45.174531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81961) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4642b8e81dd64166abc35efc1d42bbd2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4642b8e81dd64166abc35efc1d42bbd2. Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application conn.query(q) Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 01:07:45.209480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application result.read() Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 23 01:07:45.211076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application next(self.gen) Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application yield resource Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application next(self.gen) Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self.session.rollback() Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self.force_reraise() Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise self.value Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application session.commit() Sep 23 01:07:45.214591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._prepare_impl() Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self.session.flush() Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._flush(objects) Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application compat.raise_( Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application flush_context.execute() Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 23 01:07:45.217773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rec.execute(self) Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 23 01:07:45.223212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application conn.query(q) Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application result.read() Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 01:07:45.224581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [parameters: {'id': '4642b8e81dd64166abc35efc1d42bbd2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:45.226078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:45.227303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:45.229036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:45.229036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 23 01:07:45.229036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 23 01:07:45.229036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:45.229036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:45.229036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 23 01:07:45.229036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 23 01:07:45.229036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4642b8e81dd64166abc35efc1d42bbd2. Sep 23 01:07:45.229036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:45.229036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 455/932] 10.222.0.64 () {60 vars in 1140 bytes} [Fri Sep 23 01:07:44 2022] POST /identity/v3/projects => generated 262 bytes in 327 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:45.230226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cad9c9-843c-4120-86b2-f3d9fbd55c9d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:45.249019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:45.275639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:45.408556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cad9c9-843c-4120-86b2-f3d9fbd55c9d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:45.457038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7b9d8-ba59-49e6-85c5-b53d7d69f14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:45.458104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7b9d8-ba59-49e6-85c5-b53d7d69f14c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:45.458510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7b9d8-ba59-49e6-85c5-b53d7d69f14c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:45.459174 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7b9d8-ba59-49e6-85c5-b53d7d69f14c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:45.466217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd4373-488a-48ec-a111-a567b344ba56 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:45.468468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7b9d8-ba59-49e6-85c5-b53d7d69f14c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:45.470082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7b9d8-ba59-49e6-85c5-b53d7d69f14c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-a4f7b9d8-ba59-49e6-85c5-b53d7d69f14c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:45.485361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:45.486887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:45.488128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:45.488128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:45.488128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:45.488128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:45.488128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:45.488128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:45.488128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:45.488128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:45.488128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 478/933] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:45.488128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7b9d8-ba59-49e6-85c5-b53d7d69f14c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:45.505611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:45.531686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768fb7d-2489-4967-8f75-aac683d88ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:45.532683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768fb7d-2489-4967-8f75-aac683d88ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:45.533112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768fb7d-2489-4967-8f75-aac683d88ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:45.533493 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3768fb7d-2489-4967-8f75-aac683d88ac1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:45.534213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768fb7d-2489-4967-8f75-aac683d88ac1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:45.536207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3768fb7d-2489-4967-8f75-aac683d88ac1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-3768fb7d-2489-4967-8f75-aac683d88ac1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:45.551689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:45.554124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:45.557147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 456/934] 10.222.0.64 () {56 vars in 1132 bytes} [Fri Sep 23 01:07:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:45.558735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768fb7d-2489-4967-8f75-aac683d88ac1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:45.599206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:45.661800 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7b9d8-ba59-49e6-85c5-b53d7d69f14c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:45.712009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a46f46-dc61-42db-a1ba-c92129fa7a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:45.712671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a46f46-dc61-42db-a1ba-c92129fa7a70 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:45.712954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a46f46-dc61-42db-a1ba-c92129fa7a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:45.713370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a46f46-dc61-42db-a1ba-c92129fa7a70 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:45.717320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3768fb7d-2489-4967-8f75-aac683d88ac1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:45.732421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a46f46-dc61-42db-a1ba-c92129fa7a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-80a46f46-dc61-42db-a1ba-c92129fa7a70 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:45.743213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:45.745141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:07:45.746838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:45.750266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 479/935] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Sep 23 01:07:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:45.751226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a46f46-dc61-42db-a1ba-c92129fa7a70 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:45.777037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ded6b-8991-44cd-8fb9-4a0b0f0b152e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:45.777705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ded6b-8991-44cd-8fb9-4a0b0f0b152e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:45.778134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ded6b-8991-44cd-8fb9-4a0b0f0b152e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:45.778562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ded6b-8991-44cd-8fb9-4a0b0f0b152e None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:45.779254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ded6b-8991-44cd-8fb9-4a0b0f0b152e None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:45.779737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ded6b-8991-44cd-8fb9-4a0b0f0b152e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:45.781943 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ded6b-8991-44cd-8fb9-4a0b0f0b152e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:45.805100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 457/936] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:45.806169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ded6b-8991-44cd-8fb9-4a0b0f0b152e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:45.915350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a46f46-dc61-42db-a1ba-c92129fa7a70 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:45.929972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:45.954202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:45.971168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0a902-7b9a-4bc8-8524-0975c8d707be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:45.973271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0a902-7b9a-4bc8-8524-0975c8d707be None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:45.974087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0a902-7b9a-4bc8-8524-0975c8d707be None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:45.975171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee0a902-7b9a-4bc8-8524-0975c8d707be None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:45.976644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0a902-7b9a-4bc8-8524-0975c8d707be None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:45.977429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0a902-7b9a-4bc8-8524-0975c8d707be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:45.980157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee0a902-7b9a-4bc8-8524-0975c8d707be None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:45.988743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 480/937] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:45.996254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0a902-7b9a-4bc8-8524-0975c8d707be None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:46.052334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ded6b-8991-44cd-8fb9-4a0b0f0b152e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:46.121550 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c962a-4483-48d9-bfd6-ec6293a83750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:46.122956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c962a-4483-48d9-bfd6-ec6293a83750 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:46.123463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c962a-4483-48d9-bfd6-ec6293a83750 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:46.123996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c962a-4483-48d9-bfd6-ec6293a83750 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:46.125789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c962a-4483-48d9-bfd6-ec6293a83750 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:46.126131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c962a-4483-48d9-bfd6-ec6293a83750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:46.131401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c962a-4483-48d9-bfd6-ec6293a83750 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:46.143328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 458/938] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Sep 23 01:07:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:46.150104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c962a-4483-48d9-bfd6-ec6293a83750 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:46.180417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee0a902-7b9a-4bc8-8524-0975c8d707be None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:46.238686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ae434-ec21-4983-9610-f205fb31a514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:46.239997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ae434-ec21-4983-9610-f205fb31a514 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:46.240517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ae434-ec21-4983-9610-f205fb31a514 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:46.241280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757ae434-ec21-4983-9610-f205fb31a514 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:46.242762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ae434-ec21-4983-9610-f205fb31a514 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:46.243270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ae434-ec21-4983-9610-f205fb31a514 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:46.244982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757ae434-ec21-4983-9610-f205fb31a514 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:46.254788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 481/939] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:46.262259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ae434-ec21-4983-9610-f205fb31a514 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:46.329172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c962a-4483-48d9-bfd6-ec6293a83750 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:46.340290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:46.363152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:46.392413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6c838-622f-4455-bdb7-1f0ebcfeb93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:46.393528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6c838-622f-4455-bdb7-1f0ebcfeb93a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:46.394019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6c838-622f-4455-bdb7-1f0ebcfeb93a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:46.394483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6c838-622f-4455-bdb7-1f0ebcfeb93a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:46.416026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6c838-622f-4455-bdb7-1f0ebcfeb93a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:46.419438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6c838-622f-4455-bdb7-1f0ebcfeb93a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:46.433942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757ae434-ec21-4983-9610-f205fb31a514 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-c0b6c838-622f-4455-bdb7-1f0ebcfeb93a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:46.446960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:46.448701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:46.450183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:46.450183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:46.450183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:46.454658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 459/940] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:46.456941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6c838-622f-4455-bdb7-1f0ebcfeb93a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:46.513225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da700fe-8f22-4a65-84df-128e34c7c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:46.514618 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da700fe-8f22-4a65-84df-128e34c7c09e None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:46.515843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da700fe-8f22-4a65-84df-128e34c7c09e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:46.516992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da700fe-8f22-4a65-84df-128e34c7c09e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:46.519901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da700fe-8f22-4a65-84df-128e34c7c09e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:46.521855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da700fe-8f22-4a65-84df-128e34c7c09e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:46.625835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6c838-622f-4455-bdb7-1f0ebcfeb93a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:46.636195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 482/941] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:07:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:07:46.686365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1fec6-5d55-4046-83a7-c7ef67d1d416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:46.687529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1fec6-5d55-4046-83a7-c7ef67d1d416 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:46.688037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1fec6-5d55-4046-83a7-c7ef67d1d416 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:46.688505 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1fec6-5d55-4046-83a7-c7ef67d1d416 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:46.689688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1fec6-5d55-4046-83a7-c7ef67d1d416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:46.690209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1fec6-5d55-4046-83a7-c7ef67d1d416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:46.692893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1fec6-5d55-4046-83a7-c7ef67d1d416 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:46.720330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 460/942] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:46.726003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da700fe-8f22-4a65-84df-128e34c7c09e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:46.873440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da700fe-8f22-4a65-84df-128e34c7c09e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:46.888134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:46.934393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc5637-e2cf-4881-8f32-d50e755ca709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:46.935104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc5637-e2cf-4881-8f32-d50e755ca709 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:46.935385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc5637-e2cf-4881-8f32-d50e755ca709 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:46.935698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc5637-e2cf-4881-8f32-d50e755ca709 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:46.945805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc5637-e2cf-4881-8f32-d50e755ca709 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:46.947298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc5637-e2cf-4881-8f32-d50e755ca709 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-bafc5637-e2cf-4881-8f32-d50e755ca709 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:46.962588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:46.964508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:46.966054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 483/943] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Sep 23 01:07:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:46.971835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1fec6-5d55-4046-83a7-c7ef67d1d416 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:46.992330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:47.105495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0e02-0cd6-49db-a95b-11fe70c2df10 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:47.106007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0e02-0cd6-49db-a95b-11fe70c2df10 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:47.106720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c0e02-0cd6-49db-a95b-11fe70c2df10 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:47.108227 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1fec6-5d55-4046-83a7-c7ef67d1d416 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:47.111677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 484/944] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:07:47.120160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74c39f-e75a-424d-b0a2-1144c5ddde2c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:47.120663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74c39f-e75a-424d-b0a2-1144c5ddde2c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:47.121164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74c39f-e75a-424d-b0a2-1144c5ddde2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:47.153559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe5ec6-507d-48c9-9b53-f6c49a9e5489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:47.154157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe5ec6-507d-48c9-9b53-f6c49a9e5489 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:47.154583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe5ec6-507d-48c9-9b53-f6c49a9e5489 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:47.154973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe5ec6-507d-48c9-9b53-f6c49a9e5489 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:47.156073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe5ec6-507d-48c9-9b53-f6c49a9e5489 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:47.156196 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ce74c39f-e75a-424d-b0a2-1144c5ddde2c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:47.156402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe5ec6-507d-48c9-9b53-f6c49a9e5489 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:47.157959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe5ec6-507d-48c9-9b53-f6c49a9e5489 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:47.158419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ce74c39f-e75a-424d-b0a2-1144c5ddde2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:47.181051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 461/945] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Sep 23 01:07:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:47.188957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe5ec6-507d-48c9-9b53-f6c49a9e5489 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:47.256374 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 485/946] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:47.308702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:47.346415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe5ec6-507d-48c9-9b53-f6c49a9e5489 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:47.386824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc2bff-8355-4e7a-ad3b-3bf56f701e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:47.387975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc2bff-8355-4e7a-ad3b-3bf56f701e04 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:47.388520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc2bff-8355-4e7a-ad3b-3bf56f701e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:47.389273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc2bff-8355-4e7a-ad3b-3bf56f701e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:47.404467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc2bff-8355-4e7a-ad3b-3bf56f701e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:47.406549 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc2bff-8355-4e7a-ad3b-3bf56f701e04 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-28fc2bff-8355-4e7a-ad3b-3bf56f701e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:47.425017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:47.426085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:47.427036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:47.427036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:47.427036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:47.429530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 462/947] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:47.435783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74c39f-e75a-424d-b0a2-1144c5ddde2c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:47.566216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:47.570278 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74c39f-e75a-424d-b0a2-1144c5ddde2c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:47.613434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afb1e3-c981-48bc-a7b9-a4d08f43ff39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:47.614195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afb1e3-c981-48bc-a7b9-a4d08f43ff39 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:47.614475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afb1e3-c981-48bc-a7b9-a4d08f43ff39 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:47.614750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afb1e3-c981-48bc-a7b9-a4d08f43ff39 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:47.615675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afb1e3-c981-48bc-a7b9-a4d08f43ff39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:47.615989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afb1e3-c981-48bc-a7b9-a4d08f43ff39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:47.618289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afb1e3-c981-48bc-a7b9-a4d08f43ff39 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:47.634052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 486/948] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:47.641344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc2bff-8355-4e7a-ad3b-3bf56f701e04 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:47.679784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afb1e3-c981-48bc-a7b9-a4d08f43ff39 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:47.770108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc2bff-8355-4e7a-ad3b-3bf56f701e04 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:47.809828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:47.851467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca302047-1645-49d4-81f1-b2bf3cc35fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:47.853133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca302047-1645-49d4-81f1-b2bf3cc35fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:47.853990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca302047-1645-49d4-81f1-b2bf3cc35fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:47.854768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca302047-1645-49d4-81f1-b2bf3cc35fd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:47.869769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca302047-1645-49d4-81f1-b2bf3cc35fd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:47.873280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca302047-1645-49d4-81f1-b2bf3cc35fd4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:47.886928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afb1e3-c981-48bc-a7b9-a4d08f43ff39 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-ca302047-1645-49d4-81f1-b2bf3cc35fd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:47.899933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:47.901748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:47.903631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:47.903631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:47.903631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:47.903631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:47.903631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:47.903631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:47.903631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:47.903631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:47.909182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 463/949] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:47.923263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca302047-1645-49d4-81f1-b2bf3cc35fd4 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:47.946549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1984-817b-4b25-8a97-4193f33203e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:47.947187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1984-817b-4b25-8a97-4193f33203e3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:47.947476 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1984-817b-4b25-8a97-4193f33203e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:47.947873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b1984-817b-4b25-8a97-4193f33203e3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:47.948511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b1984-817b-4b25-8a97-4193f33203e3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:47.950114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671b1984-817b-4b25-8a97-4193f33203e3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-671b1984-817b-4b25-8a97-4193f33203e3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:47.963762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:47.965818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:47.967956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:47.971595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 487/950] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:47.973601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1984-817b-4b25-8a97-4193f33203e3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:48.007761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:48.130101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b1984-817b-4b25-8a97-4193f33203e3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:48.145472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca302047-1645-49d4-81f1-b2bf3cc35fd4 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:48.175386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2fc6d-137d-4dce-b54f-841cdffd7101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:48.176352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2fc6d-137d-4dce-b54f-841cdffd7101 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:48.176792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2fc6d-137d-4dce-b54f-841cdffd7101 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:48.177256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2fc6d-137d-4dce-b54f-841cdffd7101 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:48.178245 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2fc6d-137d-4dce-b54f-841cdffd7101 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:48.178744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2fc6d-137d-4dce-b54f-841cdffd7101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:48.180874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2fc6d-137d-4dce-b54f-841cdffd7101 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:48.195253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 488/951] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Sep 23 01:07:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:48.200190 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2fc6d-137d-4dce-b54f-841cdffd7101 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:48.223523 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bceb0-e9f0-4d0c-8fed-835c33e5704e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:48.224842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bceb0-e9f0-4d0c-8fed-835c33e5704e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:48.225457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bceb0-e9f0-4d0c-8fed-835c33e5704e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:48.226062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bceb0-e9f0-4d0c-8fed-835c33e5704e None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:48.228364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bceb0-e9f0-4d0c-8fed-835c33e5704e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:48.229080 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bceb0-e9f0-4d0c-8fed-835c33e5704e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:48.233003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bceb0-e9f0-4d0c-8fed-835c33e5704e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:48.263126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 464/952] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:48.275731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bceb0-e9f0-4d0c-8fed-835c33e5704e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:48.368965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:48.397058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:48.419833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2fc6d-137d-4dce-b54f-841cdffd7101 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:48.486472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bceb0-e9f0-4d0c-8fed-835c33e5704e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:48.488571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c6623-5be1-480d-a7d6-7e8b6bf6edc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:48.490628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c6623-5be1-480d-a7d6-7e8b6bf6edc5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:48.491491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c6623-5be1-480d-a7d6-7e8b6bf6edc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:48.492255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c6623-5be1-480d-a7d6-7e8b6bf6edc5 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:48.493726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c6623-5be1-480d-a7d6-7e8b6bf6edc5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:48.494498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c6623-5be1-480d-a7d6-7e8b6bf6edc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:48.497182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c6623-5be1-480d-a7d6-7e8b6bf6edc5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:48.514358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 489/953] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Sep 23 01:07:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:48.517568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c6623-5be1-480d-a7d6-7e8b6bf6edc5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:48.559067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1e2c0-187a-436b-a725-32a0b541340f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:48.560246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1e2c0-187a-436b-a725-32a0b541340f None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:48.560740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1e2c0-187a-436b-a725-32a0b541340f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:48.561234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1e2c0-187a-436b-a725-32a0b541340f None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/fa998fcd51b3404aab5412323f86908f/roles/81d0f75541e3413d802620f47d89ed62` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:48.565880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1e2c0-187a-436b-a725-32a0b541340f None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=fa998fcd51b3404aab5412323f86908f, role_id=81d0f75541e3413d802620f47d89ed62)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:48.569327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1e2c0-187a-436b-a725-32a0b541340f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:48.685325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 465/954] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:07:48 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/fa998fcd51b3404aab5412323f86908f/roles/81d0f75541e3413d802620f47d89ed62 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:07:48.728660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c6623-5be1-480d-a7d6-7e8b6bf6edc5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:48.798362 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74282a9-c202-42f8-a987-e093d284740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:48.800005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74282a9-c202-42f8-a987-e093d284740a None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:48.800801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74282a9-c202-42f8-a987-e093d284740a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:48.801551 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74282a9-c202-42f8-a987-e093d284740a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:48.803125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74282a9-c202-42f8-a987-e093d284740a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:48.806055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74282a9-c202-42f8-a987-e093d284740a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:48.809558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506bf30-5ccd-41b3-b024-b67a5fd9c99e None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:48.810443 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506bf30-5ccd-41b3-b024-b67a5fd9c99e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:48.811198 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506bf30-5ccd-41b3-b024-b67a5fd9c99e None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:48.814533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 466/955] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:48.821031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107cc9b-ea1d-4046-a709-9a7c844e06e3 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:48.821314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107cc9b-ea1d-4046-a709-9a7c844e06e3 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:48.821638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107cc9b-ea1d-4046-a709-9a7c844e06e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:48.851000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-2107cc9b-ea1d-4046-a709-9a7c844e06e3 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:48.853150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2107cc9b-ea1d-4046-a709-9a7c844e06e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:48.932062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 490/956] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:07:48 2022] POST /identity/v3/endpoints => generated 349 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:07:48.984452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 467/957] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:49.447348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74282a9-c202-42f8-a987-e093d284740a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:49.462299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:49.578658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74282a9-c202-42f8-a987-e093d284740a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:49.626469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6de9a-6437-49d1-bb7d-7e25e93a5648 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:49.627065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6de9a-6437-49d1-bb7d-7e25e93a5648 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:49.627065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c6de9a-6437-49d1-bb7d-7e25e93a5648 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:49.628721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 468/958] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:49.634848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad33d0-6804-49c6-aab9-f0b1a0ee5384 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:49.635215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad33d0-6804-49c6-aab9-f0b1a0ee5384 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:49.635509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad33d0-6804-49c6-aab9-f0b1a0ee5384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:49.650917 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:49.652452 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:49.652954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:49.653430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:49.654659 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:49.656596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ccad33d0-6804-49c6-aab9-f0b1a0ee5384 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:49.657546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:49.658011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ccad33d0-6804-49c6-aab9-f0b1a0ee5384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:49.663473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WARNING keystone.server.flask.common [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 23 01:07:49.705578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:49.764048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.sql.core [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 23 01:07:49.764048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 23 01:07:49.764048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [parameters: {'user_id': '3879b2d8410e406b8f235b2de893533b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 23 01:07:49.764048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81960) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 23 01:07:49.777865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 469/959] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application conn.query(q) Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application result.read() Sep 23 01:07:49.783106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 01:07:49.784980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application next(self.gen) Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application yield resource Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application next(self.gen) Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self.session.rollback() Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self.force_reraise() Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise self.value Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application session.commit() Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 23 01:07:49.787636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._prepare_impl() Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self.session.flush() Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._flush(objects) Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application compat.raise_( Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application flush_context.execute() Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rec.execute(self) Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 23 01:07:49.791193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 23 01:07:49.794959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application result = self._query(query) Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application conn.query(q) Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application result.read() Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 23 01:07:49.801536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [parameters: {'user_id': '3879b2d8410e406b8f235b2de893533b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:49.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 23 01:07:49.819787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 23 01:07:49.822500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 23 01:07:49.822500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 23 01:07:49.822500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 23 01:07:49.822500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 23 01:07:49.822500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 23 01:07:49.822500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:49.840219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 491/960] 10.222.0.64 () {60 vars in 1131 bytes} [Fri Sep 23 01:07:49 2022] POST /identity/v3/users => generated 160 bytes in 393 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:49.846277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad33d0-6804-49c6-aab9-f0b1a0ee5384 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:50.063080 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad33d0-6804-49c6-aab9-f0b1a0ee5384 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:50.118400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:50.141012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46b3f9-c18e-4145-bed6-71027a7824c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:50.141664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46b3f9-c18e-4145-bed6-71027a7824c7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:50.141939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46b3f9-c18e-4145-bed6-71027a7824c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:50.142215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46b3f9-c18e-4145-bed6-71027a7824c7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:50.161474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46b3f9-c18e-4145-bed6-71027a7824c7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:50.164925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46b3f9-c18e-4145-bed6-71027a7824c7 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-2a46b3f9-c18e-4145-bed6-71027a7824c7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:50.181543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:50.189316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:50.191291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 470/961] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:07:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:50.195348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46b3f9-c18e-4145-bed6-71027a7824c7 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:50.248803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:50.270915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:50.328866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8465f-c2b4-4cd1-8423-e80c2614c64e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:50.394764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7ced7-fc3d-430d-a04b-d8905311742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:50.395919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7ced7-fc3d-430d-a04b-d8905311742d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:50.396483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7ced7-fc3d-430d-a04b-d8905311742d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:50.396776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46b3f9-c18e-4145-bed6-71027a7824c7 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:50.397087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7ced7-fc3d-430d-a04b-d8905311742d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:50.397872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7ced7-fc3d-430d-a04b-d8905311742d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:50.399812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7ced7-fc3d-430d-a04b-d8905311742d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-8af7ced7-fc3d-430d-a04b-d8905311742d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:50.420318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:50.421607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:50.422563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 492/962] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:50.432743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7ced7-fc3d-430d-a04b-d8905311742d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:50.468712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d0b84-f873-44b9-a546-894a75fcc35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:50.469510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d0b84-f873-44b9-a546-894a75fcc35b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:50.469816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d0b84-f873-44b9-a546-894a75fcc35b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:50.470176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d0b84-f873-44b9-a546-894a75fcc35b None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:50.471131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d0b84-f873-44b9-a546-894a75fcc35b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:50.471448 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d0b84-f873-44b9-a546-894a75fcc35b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:50.472942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d0b84-f873-44b9-a546-894a75fcc35b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:50.523873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 471/963] 10.222.0.64 () {56 vars in 1101 bytes} [Fri Sep 23 01:07:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:50.604160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7ced7-fc3d-430d-a04b-d8905311742d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:50.656203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64beb3c7-422b-469d-b3c3-d2621cf3f0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:50.657864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64beb3c7-422b-469d-b3c3-d2621cf3f0ba None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:50.658695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64beb3c7-422b-469d-b3c3-d2621cf3f0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:50.659325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64beb3c7-422b-469d-b3c3-d2621cf3f0ba None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:50.660468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64beb3c7-422b-469d-b3c3-d2621cf3f0ba None admin] RBAC: Adding query filter params (name=compute) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:50.661035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64beb3c7-422b-469d-b3c3-d2621cf3f0ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:50.662998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64beb3c7-422b-469d-b3c3-d2621cf3f0ba None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:50.666779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:50.677290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 493/964] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:50.683217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d0b84-f873-44b9-a546-894a75fcc35b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:50.796927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:50.822735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d0b84-f873-44b9-a546-894a75fcc35b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:50.876567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b188a2d-ddf3-4bd9-8319-ee0ac3488ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:50.877354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188a2d-ddf3-4bd9-8319-ee0ac3488ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:50.877762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188a2d-ddf3-4bd9-8319-ee0ac3488ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:50.878124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188a2d-ddf3-4bd9-8319-ee0ac3488ad2 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:50.879077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b188a2d-ddf3-4bd9-8319-ee0ac3488ad2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:50.879577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b188a2d-ddf3-4bd9-8319-ee0ac3488ad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:50.881880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b188a2d-ddf3-4bd9-8319-ee0ac3488ad2 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:50.892318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 472/965] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:51.553933 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff667489-9225-4e78-8629-3061092c8155 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:51.554449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff667489-9225-4e78-8629-3061092c8155 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:51.554888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff667489-9225-4e78-8629-3061092c8155 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:51.557467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 494/966] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:51.568075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5e258-a44a-41d1-bbe7-c7e970c08410 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:51.568554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5e258-a44a-41d1-bbe7-c7e970c08410 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:51.569290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5e258-a44a-41d1-bbe7-c7e970c08410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:51.601747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-fbc5e258-a44a-41d1-bbe7-c7e970c08410 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:51.603909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fbc5e258-a44a-41d1-bbe7-c7e970c08410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:51.651368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e0e11-3dd6-4211-b684-bb6dd824737f None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:51.652321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e0e11-3dd6-4211-b684-bb6dd824737f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:51.652321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e0e11-3dd6-4211-b684-bb6dd824737f None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:51.654450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 495/967] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:51.663712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ff24e-5207-4f1e-9ea1-c7fe5eff2606 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:51.664180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ff24e-5207-4f1e-9ea1-c7fe5eff2606 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:51.664769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ff24e-5207-4f1e-9ea1-c7fe5eff2606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:51.692217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d00ff24e-5207-4f1e-9ea1-c7fe5eff2606 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:51.693826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d00ff24e-5207-4f1e-9ea1-c7fe5eff2606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:51.711956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 473/968] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:51.783599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 496/969] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:52.064231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5e258-a44a-41d1-bbe7-c7e970c08410 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:52.145156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:52.179088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5e258-a44a-41d1-bbe7-c7e970c08410 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:52.220206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd94b-4c94-49e3-882b-0eddf2af43d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:52.220868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd94b-4c94-49e3-882b-0eddf2af43d7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:52.221152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd94b-4c94-49e3-882b-0eddf2af43d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:52.221432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffd94b-4c94-49e3-882b-0eddf2af43d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:52.234590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ffd94b-4c94-49e3-882b-0eddf2af43d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:07:52.237214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ff24e-5207-4f1e-9ea1-c7fe5eff2606 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-88ffd94b-4c94-49e3-882b-0eddf2af43d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:07:52.239715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:52.240899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:07:52.241952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:52.243897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 474/970] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Sep 23 01:07:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:52.247209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd94b-4c94-49e3-882b-0eddf2af43d7 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:52.375319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ff24e-5207-4f1e-9ea1-c7fe5eff2606 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:52.388638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffd94b-4c94-49e3-882b-0eddf2af43d7 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:52.408025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:52.417684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fda1cc-0838-4eec-af2f-3a19b18e2371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:52.418394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fda1cc-0838-4eec-af2f-3a19b18e2371 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:52.425613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fda1cc-0838-4eec-af2f-3a19b18e2371 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:52.425613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fda1cc-0838-4eec-af2f-3a19b18e2371 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:52.425613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fda1cc-0838-4eec-af2f-3a19b18e2371 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:52.425613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fda1cc-0838-4eec-af2f-3a19b18e2371 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-d1fda1cc-0838-4eec-af2f-3a19b18e2371 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:52.432394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:52.433619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:52.434764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 497/971] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:52.444440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fda1cc-0838-4eec-af2f-3a19b18e2371 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:52.445700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:52.451095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe7aa-24ad-4935-8743-925d3212ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:52.452079 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe7aa-24ad-4935-8743-925d3212ac3c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:52.452552 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe7aa-24ad-4935-8743-925d3212ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:52.452868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbe7aa-24ad-4935-8743-925d3212ac3c None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:52.453887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbe7aa-24ad-4935-8743-925d3212ac3c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:52.454379 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbe7aa-24ad-4935-8743-925d3212ac3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:52.456621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbe7aa-24ad-4935-8743-925d3212ac3c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:52.472089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 475/972] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Sep 23 01:07:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:52.474567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe7aa-24ad-4935-8743-925d3212ac3c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:52.590527 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fda1cc-0838-4eec-af2f-3a19b18e2371 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:52.633323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbe7aa-24ad-4935-8743-925d3212ac3c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:52.638168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9a55-8b3e-4c53-934a-0590400f2745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:52.639041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9a55-8b3e-4c53-934a-0590400f2745 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:52.639480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9a55-8b3e-4c53-934a-0590400f2745 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:52.640595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f9a55-8b3e-4c53-934a-0590400f2745 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:52.641484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9a55-8b3e-4c53-934a-0590400f2745 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:52.641960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9a55-8b3e-4c53-934a-0590400f2745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:52.643786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f9a55-8b3e-4c53-934a-0590400f2745 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:52.653971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 498/973] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Sep 23 01:07:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:52.665720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9a55-8b3e-4c53-934a-0590400f2745 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:52.680457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7d7d3-1a60-4f2d-854d-09e574980b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:52.681165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7d7d3-1a60-4f2d-854d-09e574980b9c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:52.681446 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7d7d3-1a60-4f2d-854d-09e574980b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:52.681717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7d7d3-1a60-4f2d-854d-09e574980b9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:52.695391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7d7d3-1a60-4f2d-854d-09e574980b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:52.697078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7d7d3-1a60-4f2d-854d-09e574980b9c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-50f7d7d3-1a60-4f2d-854d-09e574980b9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:52.722775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:52.725364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:52.727163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:52.727163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:52.727163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:52.727163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 476/974] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:52.732611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7d7d3-1a60-4f2d-854d-09e574980b9c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:52.769702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:52.802852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:52.846322 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f9a55-8b3e-4c53-934a-0590400f2745 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:52.910978 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3fbf2-b4b5-4a39-98b0-6a10988e0ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:52.912093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3fbf2-b4b5-4a39-98b0-6a10988e0ffc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:52.912575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3fbf2-b4b5-4a39-98b0-6a10988e0ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:52.913059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3fbf2-b4b5-4a39-98b0-6a10988e0ffc None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:52.914239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3fbf2-b4b5-4a39-98b0-6a10988e0ffc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:52.914800 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3fbf2-b4b5-4a39-98b0-6a10988e0ffc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:52.917505 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3fbf2-b4b5-4a39-98b0-6a10988e0ffc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:52.918422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7d7d3-1a60-4f2d-854d-09e574980b9c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:52.927911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 499/975] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Sep 23 01:07:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:52.938519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3fbf2-b4b5-4a39-98b0-6a10988e0ffc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:52.985502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93475af9-daf9-45f8-b1c2-158e60e6b361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:52.986556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93475af9-daf9-45f8-b1c2-158e60e6b361 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:52.987056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93475af9-daf9-45f8-b1c2-158e60e6b361 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:52.987586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93475af9-daf9-45f8-b1c2-158e60e6b361 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:52.988847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93475af9-daf9-45f8-b1c2-158e60e6b361 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:52.989406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93475af9-daf9-45f8-b1c2-158e60e6b361 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:52.992106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93475af9-daf9-45f8-b1c2-158e60e6b361 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:53.023606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 477/976] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:53.025865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93475af9-daf9-45f8-b1c2-158e60e6b361 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:53.122739 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3fbf2-b4b5-4a39-98b0-6a10988e0ffc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:53.158413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:53.194422 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5f343-b314-46fb-bc3d-52c25c09a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:53.194422 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5f343-b314-46fb-bc3d-52c25c09a6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:53.194422 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5f343-b314-46fb-bc3d-52c25c09a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:53.194422 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5f343-b314-46fb-bc3d-52c25c09a6f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:53.195153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5f343-b314-46fb-bc3d-52c25c09a6f9 None admin] RBAC: Adding query filter params (interface=admin, service_id=f5150fe704124e19bebce65d1991c813, region_id=RegionOne) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:53.195153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5f343-b314-46fb-bc3d-52c25c09a6f9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:53.195153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5f343-b314-46fb-bc3d-52c25c09a6f9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:53.207165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 500/977] 10.222.0.64 () {56 vars in 1249 bytes} [Fri Sep 23 01:07:52 2022] GET /identity/v3/endpoints?service_id=f5150fe704124e19bebce65d1991c813&interface=admin®ion_id=RegionOne => generated 193 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:53.207425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93475af9-daf9-45f8-b1c2-158e60e6b361 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:53.209404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5f343-b314-46fb-bc3d-52c25c09a6f9 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:53.220148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:53.258803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0628e-7345-4e3c-b651-0b0f21854f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:53.259965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0628e-7345-4e3c-b651-0b0f21854f53 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:53.260280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0628e-7345-4e3c-b651-0b0f21854f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:53.260558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0628e-7345-4e3c-b651-0b0f21854f53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:53.272103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0628e-7345-4e3c-b651-0b0f21854f53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:53.274816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0628e-7345-4e3c-b651-0b0f21854f53 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-2ae0628e-7345-4e3c-b651-0b0f21854f53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:07:53.293085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:53.294772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:53.297719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 478/978] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Sep 23 01:07:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:53.305157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0628e-7345-4e3c-b651-0b0f21854f53 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:53.414270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5f343-b314-46fb-bc3d-52c25c09a6f9 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:53.486664 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183b623-0e30-4d76-b372-a55918a870c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:53.487730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183b623-0e30-4d76-b372-a55918a870c1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:53.488240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183b623-0e30-4d76-b372-a55918a870c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:53.488753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183b623-0e30-4d76-b372-a55918a870c1 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:53.490058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183b623-0e30-4d76-b372-a55918a870c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:53.492646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183b623-0e30-4d76-b372-a55918a870c1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:53.502826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0628e-7345-4e3c-b651-0b0f21854f53 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:53.511702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 501/979] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Sep 23 01:07:53 2022] GET /identity/v3/services => generated 1497 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:07:53.511702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0891a-51ca-49ed-8954-0d793608d644 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:53.511702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0891a-51ca-49ed-8954-0d793608d644 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:53.511702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0891a-51ca-49ed-8954-0d793608d644 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:53.517515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 502/980] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:53.520609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8424e-369a-4c36-96a7-c2ffba83ef49 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:53.521109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8424e-369a-4c36-96a7-c2ffba83ef49 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:53.521622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f8424e-369a-4c36-96a7-c2ffba83ef49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:53.555574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-92f8424e-369a-4c36-96a7-c2ffba83ef49 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:53.557868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-92f8424e-369a-4c36-96a7-c2ffba83ef49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:53.580864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbda935-c18b-4029-a53b-4c4441a4fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:53.581503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbda935-c18b-4029-a53b-4c4441a4fcbe None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:53.581782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbda935-c18b-4029-a53b-4c4441a4fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:53.582059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbda935-c18b-4029-a53b-4c4441a4fcbe None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:53.582954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbda935-c18b-4029-a53b-4c4441a4fcbe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:53.583382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbda935-c18b-4029-a53b-4c4441a4fcbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:53.585525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbda935-c18b-4029-a53b-4c4441a4fcbe None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:53.611981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 479/981] 10.222.0.64 () {56 vars in 1149 bytes} [Fri Sep 23 01:07:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:53.615695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbda935-c18b-4029-a53b-4c4441a4fcbe None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:53.648958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:53.658398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:53.703206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 503/982] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:53.785103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbda935-c18b-4029-a53b-4c4441a4fcbe None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:53.831314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc15655-1a8d-46e0-aba3-0d0b987cc7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:53.832142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15655-1a8d-46e0-aba3-0d0b987cc7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:53.832452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15655-1a8d-46e0-aba3-0d0b987cc7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:53.832751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc15655-1a8d-46e0-aba3-0d0b987cc7c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:53.929436 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc15655-1a8d-46e0-aba3-0d0b987cc7c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:53.930913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc15655-1a8d-46e0-aba3-0d0b987cc7c5 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:53.959531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:54.057750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 480/983] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:07:53 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:07:54.065601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8424e-369a-4c36-96a7-c2ffba83ef49 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:54.148576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc15655-1a8d-46e0-aba3-0d0b987cc7c5 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:54.191577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f8424e-369a-4c36-96a7-c2ffba83ef49 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:54.244137 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48432a8-e0a7-417d-b55a-03c1ae9340be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:54.245130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48432a8-e0a7-417d-b55a-03c1ae9340be None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:54.245636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48432a8-e0a7-417d-b55a-03c1ae9340be None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:54.246125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48432a8-e0a7-417d-b55a-03c1ae9340be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:54.267030 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48432a8-e0a7-417d-b55a-03c1ae9340be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:54.269874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48432a8-e0a7-417d-b55a-03c1ae9340be None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-e48432a8-e0a7-417d-b55a-03c1ae9340be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:07:54.297970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:54.300000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:07:54.301653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:07:54.301653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:07:54.301653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:54.301653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 504/984] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:54.309503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48432a8-e0a7-417d-b55a-03c1ae9340be None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:54.345646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc15655-1a8d-46e0-aba3-0d0b987cc7c5 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:54.365704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:54.391479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:54.409665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c924281-9a33-4a6f-8124-bd93ffb9fae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:54.410520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c924281-9a33-4a6f-8124-bd93ffb9fae1 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:54.410909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c924281-9a33-4a6f-8124-bd93ffb9fae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:54.411289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c924281-9a33-4a6f-8124-bd93ffb9fae1 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:54.412279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c924281-9a33-4a6f-8124-bd93ffb9fae1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:54.414488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c924281-9a33-4a6f-8124-bd93ffb9fae1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:54.487764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48432a8-e0a7-417d-b55a-03c1ae9340be None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:54.500113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 481/985] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:07:54 2022] POST /identity/v3/services => generated 242 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:07:54.559939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08be80e-2d3a-44f7-a475-91f5f5dbbc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:54.560504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08be80e-2d3a-44f7-a475-91f5f5dbbc87 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:54.560790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08be80e-2d3a-44f7-a475-91f5f5dbbc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:54.561132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08be80e-2d3a-44f7-a475-91f5f5dbbc87 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:54.562384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08be80e-2d3a-44f7-a475-91f5f5dbbc87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:54.562777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08be80e-2d3a-44f7-a475-91f5f5dbbc87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:54.564661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08be80e-2d3a-44f7-a475-91f5f5dbbc87 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:54.594688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 505/986] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:54.596526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c924281-9a33-4a6f-8124-bd93ffb9fae1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:54.766127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c924281-9a33-4a6f-8124-bd93ffb9fae1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:54.817238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abc851-ad61-46bd-a459-30089e5c396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:54.818096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abc851-ad61-46bd-a459-30089e5c396f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:54.818235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abc851-ad61-46bd-a459-30089e5c396f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:54.818571 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93abc851-ad61-46bd-a459-30089e5c396f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:54.830263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abc851-ad61-46bd-a459-30089e5c396f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:54.832031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93abc851-ad61-46bd-a459-30089e5c396f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-93abc851-ad61-46bd-a459-30089e5c396f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:07:54.846997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:54.848940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:54.850620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:07:54.850620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:07:54.850620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:07:54.850620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:07:54.850620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:07:54.850620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:07:54.850620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:07:54.850620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:54.850620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 482/987] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:54.861294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08be80e-2d3a-44f7-a475-91f5f5dbbc87 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:55.036305 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:55.054772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08be80e-2d3a-44f7-a475-91f5f5dbbc87 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:55.106261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a742fb-8b55-4015-a244-050dca50c6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:55.107147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a742fb-8b55-4015-a244-050dca50c6ce None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:55.107688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a742fb-8b55-4015-a244-050dca50c6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:55.108134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a742fb-8b55-4015-a244-050dca50c6ce None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:55.109030 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a742fb-8b55-4015-a244-050dca50c6ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:55.109492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a742fb-8b55-4015-a244-050dca50c6ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:55.111138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a742fb-8b55-4015-a244-050dca50c6ce None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:55.125537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 506/988] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:07:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:55.131506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abc851-ad61-46bd-a459-30089e5c396f None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:55.210931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:55.318524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93abc851-ad61-46bd-a459-30089e5c396f None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:55.367365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ab9b9-952c-4ca4-a8f9-3e8fad7810c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:55.368228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ab9b9-952c-4ca4-a8f9-3e8fad7810c6 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:55.368554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ab9b9-952c-4ca4-a8f9-3e8fad7810c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:55.368853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ab9b9-952c-4ca4-a8f9-3e8fad7810c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:55.369792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ab9b9-952c-4ca4-a8f9-3e8fad7810c6 None admin] RBAC: Adding query filter params (role.id=81d0f75541e3413d802620f47d89ed62, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=fa998fcd51b3404aab5412323f86908f) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:55.370127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ab9b9-952c-4ca4-a8f9-3e8fad7810c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:55.371761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ab9b9-952c-4ca4-a8f9-3e8fad7810c6 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:55.387895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 483/989] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:07:55 2022] GET /identity/v3/role_assignments?user.id=fa998fcd51b3404aab5412323f86908f&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=81d0f75541e3413d802620f47d89ed62 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:56.293165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880bb95-d5a2-4a28-96c4-17bfa3ea3c08 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:56.294001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880bb95-d5a2-4a28-96c4-17bfa3ea3c08 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:56.294001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880bb95-d5a2-4a28-96c4-17bfa3ea3c08 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:56.295616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 507/990] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:07:56.302261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78650c-f333-4d1a-803b-fd0097e6f882 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:56.303064 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78650c-f333-4d1a-803b-fd0097e6f882 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:56.303064 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78650c-f333-4d1a-803b-fd0097e6f882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:56.325962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-be78650c-f333-4d1a-803b-fd0097e6f882 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:56.327824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-be78650c-f333-4d1a-803b-fd0097e6f882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:56.415684 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 484/991] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:07:56.701886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880bb95-d5a2-4a28-96c4-17bfa3ea3c08 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:56.842263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880bb95-d5a2-4a28-96c4-17bfa3ea3c08 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:56.888910 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae318596-f400-4d75-b814-2026fb1d500a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:56.889986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae318596-f400-4d75-b814-2026fb1d500a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:56.890465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae318596-f400-4d75-b814-2026fb1d500a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:56.890973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae318596-f400-4d75-b814-2026fb1d500a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:56.891986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae318596-f400-4d75-b814-2026fb1d500a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:56.893765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae318596-f400-4d75-b814-2026fb1d500a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-ae318596-f400-4d75-b814-2026fb1d500a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:56.908574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:56.910580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:56.912363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 508/992] 10.222.0.64 () {56 vars in 1126 bytes} [Fri Sep 23 01:07:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:56.920470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78650c-f333-4d1a-803b-fd0097e6f882 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:56.967810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:57.004172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c88cf-e2e9-46ed-8f35-14d54d151b2f None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:57.004663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c88cf-e2e9-46ed-8f35-14d54d151b2f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:57.005149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c88cf-e2e9-46ed-8f35-14d54d151b2f None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:57.008642 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 509/993] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:07:57.017682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b627b8-653f-4a68-acb5-b35d3a80e280 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:57.018164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b627b8-653f-4a68-acb5-b35d3a80e280 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:57.018627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b627b8-653f-4a68-acb5-b35d3a80e280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:57.044595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78650c-f333-4d1a-803b-fd0097e6f882 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:57.051376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f9b627b8-653f-4a68-acb5-b35d3a80e280 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:57.054711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f9b627b8-653f-4a68-acb5-b35d3a80e280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:57.058872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:57.086704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5951fb-a7d0-491f-827e-714f3606ef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:57.087443 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5951fb-a7d0-491f-827e-714f3606ef55 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:57.087755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5951fb-a7d0-491f-827e-714f3606ef55 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:57.088032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5951fb-a7d0-491f-827e-714f3606ef55 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:57.088752 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5951fb-a7d0-491f-827e-714f3606ef55 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:57.089072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5951fb-a7d0-491f-827e-714f3606ef55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:57.090463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5951fb-a7d0-491f-827e-714f3606ef55 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:57.100796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 485/994] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Sep 23 01:07:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:57.102653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5951fb-a7d0-491f-827e-714f3606ef55 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:57.167013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 510/995] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:57.238325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5951fb-a7d0-491f-827e-714f3606ef55 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:57.241412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:57.307909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339cd4d-6491-437e-91f8-68b879744e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:57.309004 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339cd4d-6491-437e-91f8-68b879744e77 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:57.309504 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339cd4d-6491-437e-91f8-68b879744e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:57.310012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339cd4d-6491-437e-91f8-68b879744e77 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:57.311182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339cd4d-6491-437e-91f8-68b879744e77 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:57.311814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339cd4d-6491-437e-91f8-68b879744e77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:57.314451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339cd4d-6491-437e-91f8-68b879744e77 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:57.325089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 486/996] 10.222.0.64 () {56 vars in 1122 bytes} [Fri Sep 23 01:07:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:07:57.334272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b627b8-653f-4a68-acb5-b35d3a80e280 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:57.531530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b627b8-653f-4a68-acb5-b35d3a80e280 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:57.593209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61330eb1-d349-4ed1-a724-a840b613bd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:57.594349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330eb1-d349-4ed1-a724-a840b613bd55 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:57.594839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330eb1-d349-4ed1-a724-a840b613bd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:57.595408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61330eb1-d349-4ed1-a724-a840b613bd55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:57.596653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61330eb1-d349-4ed1-a724-a840b613bd55 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:57.599366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61330eb1-d349-4ed1-a724-a840b613bd55 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:57.610820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339cd4d-6491-437e-91f8-68b879744e77 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:57.730991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:57.739444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 511/997] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:07:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:07:57.773513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339cd4d-6491-437e-91f8-68b879744e77 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:57.838288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8debd-9941-4712-9134-870b498767dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:57.841450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8debd-9941-4712-9134-870b498767dd None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:57.841450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8debd-9941-4712-9134-870b498767dd None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:57.841450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8debd-9941-4712-9134-870b498767dd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:57.841450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8debd-9941-4712-9134-870b498767dd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:57.841781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8debd-9941-4712-9134-870b498767dd None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-7ae8debd-9941-4712-9134-870b498767dd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:57.853069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:57.854345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:07:57.861617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 487/998] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:07:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:07:57.864585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61330eb1-d349-4ed1-a724-a840b613bd55 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:58.014533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61330eb1-d349-4ed1-a724-a840b613bd55 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:58.064674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23eb60-c711-494d-9cdf-544d3114286b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:58.065476 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23eb60-c711-494d-9cdf-544d3114286b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:58.065858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23eb60-c711-494d-9cdf-544d3114286b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:58.066225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23eb60-c711-494d-9cdf-544d3114286b None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:58.066999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23eb60-c711-494d-9cdf-544d3114286b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:58.067400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23eb60-c711-494d-9cdf-544d3114286b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:58.069557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23eb60-c711-494d-9cdf-544d3114286b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:58.076764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 512/999] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:58.084653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8debd-9941-4712-9134-870b498767dd None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:58.135299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:58.260603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8debd-9941-4712-9134-870b498767dd None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:58.322989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef0d69-c58a-4b80-8636-953fc9d85a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:58.323742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef0d69-c58a-4b80-8636-953fc9d85a94 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:58.324021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef0d69-c58a-4b80-8636-953fc9d85a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:58.324294 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef0d69-c58a-4b80-8636-953fc9d85a94 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:58.325204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef0d69-c58a-4b80-8636-953fc9d85a94 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:58.325538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef0d69-c58a-4b80-8636-953fc9d85a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:58.327215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef0d69-c58a-4b80-8636-953fc9d85a94 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:58.338638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 488/1000] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:07:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:58.341828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23eb60-c711-494d-9cdf-544d3114286b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:58.404345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e910e-3bbb-48ff-8506-ce12403f8503 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:58.404624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e910e-3bbb-48ff-8506-ce12403f8503 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:58.404898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132e910e-3bbb-48ff-8506-ce12403f8503 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:58.406892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 489/1001] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:07:58.412592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d02e7b-4a0f-45d9-8dcb-3c36a976892b None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:58.412877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d02e7b-4a0f-45d9-8dcb-3c36a976892b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:58.413152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d02e7b-4a0f-45d9-8dcb-3c36a976892b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:58.434019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-54d02e7b-4a0f-45d9-8dcb-3c36a976892b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:07:58.435606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-54d02e7b-4a0f-45d9-8dcb-3c36a976892b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:07:58.439155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:58.505439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23eb60-c711-494d-9cdf-544d3114286b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:58.553243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fc837-d40f-400b-b8b5-1ee88a2ffff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:58.554067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fc837-d40f-400b-b8b5-1ee88a2ffff9 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:58.554514 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fc837-d40f-400b-b8b5-1ee88a2ffff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:58.554915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361fc837-d40f-400b-b8b5-1ee88a2ffff9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:58.555914 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fc837-d40f-400b-b8b5-1ee88a2ffff9 None admin] RBAC: Adding query filter params (interface=public, service_id=23ec5a426ae246299d8028cc6a684184, region_id=RegionOne) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:58.556361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fc837-d40f-400b-b8b5-1ee88a2ffff9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:58.557801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 490/1002] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:07:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:07:58.558286 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361fc837-d40f-400b-b8b5-1ee88a2ffff9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:58.570669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 513/1003] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Sep 23 01:07:58 2022] GET /identity/v3/endpoints?service_id=23ec5a426ae246299d8028cc6a684184&interface=public®ion_id=RegionOne => generated 194 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:58.572807 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d02e7b-4a0f-45d9-8dcb-3c36a976892b None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:58.601307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:58.734246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d02e7b-4a0f-45d9-8dcb-3c36a976892b None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:58.794239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb47690-24bf-4567-aa42-936beff3915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:58.794863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb47690-24bf-4567-aa42-936beff3915e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:58.795146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb47690-24bf-4567-aa42-936beff3915e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:58.795422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb47690-24bf-4567-aa42-936beff3915e None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:58.796087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb47690-24bf-4567-aa42-936beff3915e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:58.798168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb47690-24bf-4567-aa42-936beff3915e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:58.806586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 491/1004] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Sep 23 01:07:58 2022] GET /identity/v3/services => generated 1727 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:07:58.885764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fc837-d40f-400b-b8b5-1ee88a2ffff9 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:59.079630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361fc837-d40f-400b-b8b5-1ee88a2ffff9 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:59.133379 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:59.150771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc192ac-c2d1-44f8-beff-49e8af695429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:59.150771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc192ac-c2d1-44f8-beff-49e8af695429 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:59.150771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc192ac-c2d1-44f8-beff-49e8af695429 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:59.150771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc192ac-c2d1-44f8-beff-49e8af695429 None admin] PATH_INFO: `/v3/services/image` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:59.151533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc192ac-c2d1-44f8-beff-49e8af695429 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:59.154100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc192ac-c2d1-44f8-beff-49e8af695429 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-5dc192ac-c2d1-44f8-beff-49e8af695429 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:07:59.172144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:07:59.174436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 01:07:59.175875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:07:59.183586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 514/1005] 10.222.0.64 () {56 vars in 1105 bytes} [Fri Sep 23 01:07:58 2022] GET /identity/v3/services/image => generated 86 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:07:59.187685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb47690-24bf-4567-aa42-936beff3915e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:59.295995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:07:59.316022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb47690-24bf-4567-aa42-936beff3915e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:59.363221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240ee73-81e2-4934-94eb-bd5c50662189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:59.364082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240ee73-81e2-4934-94eb-bd5c50662189 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:59.364383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240ee73-81e2-4934-94eb-bd5c50662189 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:59.364636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240ee73-81e2-4934-94eb-bd5c50662189 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:59.365341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240ee73-81e2-4934-94eb-bd5c50662189 None admin] RBAC: Adding query filter params (name=image) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:59.365646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240ee73-81e2-4934-94eb-bd5c50662189 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:59.367068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240ee73-81e2-4934-94eb-bd5c50662189 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:59.378289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 492/1006] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:07:59.379703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc192ac-c2d1-44f8-beff-49e8af695429 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:07:59.518118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc192ac-c2d1-44f8-beff-49e8af695429 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:07:59.586363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395c161-ba96-4b71-bf26-95878ebd3617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:07:59.587447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395c161-ba96-4b71-bf26-95878ebd3617 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:07:59.588176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395c161-ba96-4b71-bf26-95878ebd3617 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:07:59.588482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395c161-ba96-4b71-bf26-95878ebd3617 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:07:59.589801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395c161-ba96-4b71-bf26-95878ebd3617 None admin] RBAC: Adding query filter params (type=image) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:07:59.589979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395c161-ba96-4b71-bf26-95878ebd3617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:07:59.592562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395c161-ba96-4b71-bf26-95878ebd3617 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:07:59.607592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 515/1007] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:07:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:00.636146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2e146-eb53-4d0a-86c8-36a4dfad80c9 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:00.636990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2e146-eb53-4d0a-86c8-36a4dfad80c9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:00.637192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2e146-eb53-4d0a-86c8-36a4dfad80c9 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:00.639239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 493/1008] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:00.645504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162f007-31a4-4864-a414-13828711f73a None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:00.646328 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162f007-31a4-4864-a414-13828711f73a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:00.646470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162f007-31a4-4864-a414-13828711f73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:00.669224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d162f007-31a4-4864-a414-13828711f73a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:00.670855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d162f007-31a4-4864-a414-13828711f73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:00.748090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:00.783175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 516/1009] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:08:01.217096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2e146-eb53-4d0a-86c8-36a4dfad80c9 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:01.363056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2e146-eb53-4d0a-86c8-36a4dfad80c9 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:01.410535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c9de8-c174-45ce-908a-2774e8adb553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:01.411206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c9de8-c174-45ce-908a-2774e8adb553 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:01.411484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c9de8-c174-45ce-908a-2774e8adb553 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:01.411787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c9de8-c174-45ce-908a-2774e8adb553 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:01.412974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c9de8-c174-45ce-908a-2774e8adb553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:01.414434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c9de8-c174-45ce-908a-2774e8adb553 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:01.416462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 494/1010] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:01.423623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162f007-31a4-4864-a414-13828711f73a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:01.433104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8fc26-9329-43c9-ae86-f4af3e00ecb8 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:01.433629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8fc26-9329-43c9-ae86-f4af3e00ecb8 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:01.434124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8fc26-9329-43c9-ae86-f4af3e00ecb8 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:01.436891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 495/1011] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:08:01.445127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa78697-5385-49e0-9abb-0cd2b8488820 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:01.445567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa78697-5385-49e0-9abb-0cd2b8488820 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:01.445998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa78697-5385-49e0-9abb-0cd2b8488820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:01.477245 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-5fa78697-5385-49e0-9abb-0cd2b8488820 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:01.479010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5fa78697-5385-49e0-9abb-0cd2b8488820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:01.500219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:01.599739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 496/1012] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:01 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:08:01.607012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162f007-31a4-4864-a414-13828711f73a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:01.680807 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f66f7-9974-4132-b029-70dd6f25ba6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:01.681821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f66f7-9974-4132-b029-70dd6f25ba6b None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:01.681979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f66f7-9974-4132-b029-70dd6f25ba6b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:01.682306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f66f7-9974-4132-b029-70dd6f25ba6b None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:01.682990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f66f7-9974-4132-b029-70dd6f25ba6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:01.685717 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f66f7-9974-4132-b029-70dd6f25ba6b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:01.782396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 517/1013] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:08:01 2022] POST /identity/v3/projects => generated 311 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:08:01.962012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa78697-5385-49e0-9abb-0cd2b8488820 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:02.140673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa78697-5385-49e0-9abb-0cd2b8488820 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:02.166855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca8dff-0765-480f-97fd-acfd5a7a3242 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:02.167375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca8dff-0765-480f-97fd-acfd5a7a3242 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:02.168082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca8dff-0765-480f-97fd-acfd5a7a3242 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:02.169627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 518/1014] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:02.178236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d3750-8bb6-4715-8d91-20d3a41e051b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:02.178760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d3750-8bb6-4715-8d91-20d3a41e051b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:02.179167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d3750-8bb6-4715-8d91-20d3a41e051b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:02.200542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befd4f2-4ad4-4ce3-8768-1b28008243d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:02.201229 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd4f2-4ad4-4ce3-8768-1b28008243d7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:02.201510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd4f2-4ad4-4ce3-8768-1b28008243d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:02.201782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befd4f2-4ad4-4ce3-8768-1b28008243d7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:02.202262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3a6d3750-8bb6-4715-8d91-20d3a41e051b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:02.202926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befd4f2-4ad4-4ce3-8768-1b28008243d7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:02.204300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3a6d3750-8bb6-4715-8d91-20d3a41e051b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:02.204616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befd4f2-4ad4-4ce3-8768-1b28008243d7 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-5befd4f2-4ad4-4ce3-8768-1b28008243d7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:02.217731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:02.220074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:02.224363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 497/1015] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:08:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:02.229542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befd4f2-4ad4-4ce3-8768-1b28008243d7 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:02.252112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:02.257319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:02.323625 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 519/1016] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:08:02.423524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befd4f2-4ad4-4ce3-8768-1b28008243d7 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:02.490433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854daf3-3c30-41d3-ac3c-df3f1c03cfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:02.491506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854daf3-3c30-41d3-ac3c-df3f1c03cfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:02.492029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854daf3-3c30-41d3-ac3c-df3f1c03cfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:02.492558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854daf3-3c30-41d3-ac3c-df3f1c03cfe2 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:02.493719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854daf3-3c30-41d3-ac3c-df3f1c03cfe2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:02.494296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854daf3-3c30-41d3-ac3c-df3f1c03cfe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:02.497002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854daf3-3c30-41d3-ac3c-df3f1c03cfe2 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:02.507605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 498/1017] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:08:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:02.517516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d3750-8bb6-4715-8d91-20d3a41e051b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:02.668862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d3750-8bb6-4715-8d91-20d3a41e051b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:02.718146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6591af-b032-4845-a637-3cf026678742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:02.719069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6591af-b032-4845-a637-3cf026678742 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:02.719069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6591af-b032-4845-a637-3cf026678742 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:02.719298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6591af-b032-4845-a637-3cf026678742 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:02.719924 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6591af-b032-4845-a637-3cf026678742 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:02.720220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6591af-b032-4845-a637-3cf026678742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:02.721630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6591af-b032-4845-a637-3cf026678742 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:02.730830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 520/1018] 10.222.0.64 () {56 vars in 1112 bytes} [Fri Sep 23 01:08:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:02.737123 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854daf3-3c30-41d3-ac3c-df3f1c03cfe2 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:02.795785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6591af-b032-4845-a637-3cf026678742 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:02.851498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:02.862660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:02.906765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854daf3-3c30-41d3-ac3c-df3f1c03cfe2 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:02.957703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6591af-b032-4845-a637-3cf026678742 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:02.964414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf14fa-e6eb-434f-9a08-bdc81419eb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:02.965387 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf14fa-e6eb-434f-9a08-bdc81419eb62 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:02.965796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf14fa-e6eb-434f-9a08-bdc81419eb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:02.966206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf14fa-e6eb-434f-9a08-bdc81419eb62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:02.967084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf14fa-e6eb-434f-9a08-bdc81419eb62 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:02.968812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf14fa-e6eb-434f-9a08-bdc81419eb62 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:03.026735 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5998db-ddbd-4204-9d3b-682b9afca8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:03.027936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5998db-ddbd-4204-9d3b-682b9afca8ce None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:03.028435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5998db-ddbd-4204-9d3b-682b9afca8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:03.028938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5998db-ddbd-4204-9d3b-682b9afca8ce None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:03.030214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5998db-ddbd-4204-9d3b-682b9afca8ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:03.032991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5998db-ddbd-4204-9d3b-682b9afca8ce None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:03.071584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 499/1019] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:08:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:08:03.159646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 521/1020] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:08:02 2022] POST /identity/v3/services => generated 242 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:08:04.980652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1ead0-7a52-49e5-82f0-f428e9173c94 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:04.981351 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1ead0-7a52-49e5-82f0-f428e9173c94 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:04.981631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1ead0-7a52-49e5-82f0-f428e9173c94 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:04.984503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 500/1021] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:04.993726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a246de-9b2d-4c0b-85bc-a03e9c019f19 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:04.994601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a246de-9b2d-4c0b-85bc-a03e9c019f19 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:04.994601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a246de-9b2d-4c0b-85bc-a03e9c019f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:05.016660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-83a246de-9b2d-4c0b-85bc-a03e9c019f19 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:05.018090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-83a246de-9b2d-4c0b-85bc-a03e9c019f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:05.147282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 522/1022] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:05.455520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1ead0-7a52-49e5-82f0-f428e9173c94 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:05.596485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1ead0-7a52-49e5-82f0-f428e9173c94 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:05.638968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:05.646267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece098f-4c1f-49aa-935d-9efcc8a61fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:05.647228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece098f-4c1f-49aa-935d-9efcc8a61fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:05.647674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece098f-4c1f-49aa-935d-9efcc8a61fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:05.648070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece098f-4c1f-49aa-935d-9efcc8a61fe4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:05.649483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece098f-4c1f-49aa-935d-9efcc8a61fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:05.652757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece098f-4c1f-49aa-935d-9efcc8a61fe4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:05.653420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 501/1023] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:05.660456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a246de-9b2d-4c0b-85bc-a03e9c019f19 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:05.671053 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2be7b-770b-41b3-8aea-63efb51be5d1 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:05.672128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2be7b-770b-41b3-8aea-63efb51be5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:05.672638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2be7b-770b-41b3-8aea-63efb51be5d1 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:05.677137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 502/1024] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:08:05.684944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4c0b1-2ba8-4895-9665-e16d585d5934 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:05.685321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4c0b1-2ba8-4895-9665-e16d585d5934 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:05.685594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4c0b1-2ba8-4895-9665-e16d585d5934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:05.688150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:05.708078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ffe4c0b1-2ba8-4895-9665-e16d585d5934 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:05.709330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ffe4c0b1-2ba8-4895-9665-e16d585d5934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:05.798750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 503/1025] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:05.852867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a246de-9b2d-4c0b-85bc-a03e9c019f19 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:05.993299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee2912-39cf-462f-afc3-ab149eef2dc3 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:05.993990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee2912-39cf-462f-afc3-ab149eef2dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:05.993990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee2912-39cf-462f-afc3-ab149eef2dc3 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:05.995483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 504/1026] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:08:06.002300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1af5b6-7227-427f-b407-be2fb5137f7c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:06.002628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1af5b6-7227-427f-b407-be2fb5137f7c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:06.002934 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1af5b6-7227-427f-b407-be2fb5137f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:06.021663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb81b02-235f-4acc-94fb-8f44199e3f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:06.022758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb81b02-235f-4acc-94fb-8f44199e3f7a None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:06.023272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb81b02-235f-4acc-94fb-8f44199e3f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:06.023828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb81b02-235f-4acc-94fb-8f44199e3f7a None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:06.026210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb81b02-235f-4acc-94fb-8f44199e3f7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:06.029098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb81b02-235f-4acc-94fb-8f44199e3f7a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:06.032233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8b1af5b6-7227-427f-b407-be2fb5137f7c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:06.034275 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8b1af5b6-7227-427f-b407-be2fb5137f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:06.126664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 505/1027] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:08:06.161741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1af5b6-7227-427f-b407-be2fb5137f7c None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:06.191832 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 523/1028] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:08:05 2022] POST /identity/v3/users => generated 285 bytes in 532 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:08:06.272874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:06.333040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1af5b6-7227-427f-b407-be2fb5137f7c None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:06.381235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345633c-90bb-4367-8b0c-a5d7d4cf56f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:06.382087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345633c-90bb-4367-8b0c-a5d7d4cf56f3 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:06.382500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345633c-90bb-4367-8b0c-a5d7d4cf56f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:06.382959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345633c-90bb-4367-8b0c-a5d7d4cf56f3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:06.398084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345633c-90bb-4367-8b0c-a5d7d4cf56f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-6345633c-90bb-4367-8b0c-a5d7d4cf56f3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:08:06.404101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:06.405191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:06.408922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 506/1029] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Sep 23 01:08:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:06.416237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb81b02-235f-4acc-94fb-8f44199e3f7a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:06.475682 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345633c-90bb-4367-8b0c-a5d7d4cf56f3 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:06.561207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:06.595917 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb81b02-235f-4acc-94fb-8f44199e3f7a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:06.655173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a40f8d-b5a0-40fb-bf39-155d462c41fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:06.656946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a40f8d-b5a0-40fb-bf39-155d462c41fc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:06.657458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a40f8d-b5a0-40fb-bf39-155d462c41fc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:06.657952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a40f8d-b5a0-40fb-bf39-155d462c41fc None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:06.658509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345633c-90bb-4367-8b0c-a5d7d4cf56f3 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:06.659252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a40f8d-b5a0-40fb-bf39-155d462c41fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:06.659797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a40f8d-b5a0-40fb-bf39-155d462c41fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:06.663877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a40f8d-b5a0-40fb-bf39-155d462c41fc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:06.697491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 524/1030] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Sep 23 01:08:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:06.700903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a40f8d-b5a0-40fb-bf39-155d462c41fc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:06.705781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:06.739618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0aa40-ddef-4807-8e9b-72aca8f18828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:06.740726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0aa40-ddef-4807-8e9b-72aca8f18828 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:06.741178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0aa40-ddef-4807-8e9b-72aca8f18828 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:06.741627 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0aa40-ddef-4807-8e9b-72aca8f18828 None admin] PATH_INFO: `/v3/services/image` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:06.742637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0aa40-ddef-4807-8e9b-72aca8f18828 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:06.744986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0aa40-ddef-4807-8e9b-72aca8f18828 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-1ef0aa40-ddef-4807-8e9b-72aca8f18828 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:06.761431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:06.763450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:06.765403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 507/1031] 10.222.0.64 () {56 vars in 1105 bytes} [Fri Sep 23 01:08:06 2022] GET /identity/v3/services/image => generated 86 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:08:06.771106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0aa40-ddef-4807-8e9b-72aca8f18828 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:06.898584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a40f8d-b5a0-40fb-bf39-155d462c41fc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:06.959340 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687479d-3e82-440c-a4bd-4da2bd240184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:06.960537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687479d-3e82-440c-a4bd-4da2bd240184 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:06.961049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687479d-3e82-440c-a4bd-4da2bd240184 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:06.961524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687479d-3e82-440c-a4bd-4da2bd240184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:06.966992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:06.968109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0aa40-ddef-4807-8e9b-72aca8f18828 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:06.994064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687479d-3e82-440c-a4bd-4da2bd240184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:06.996086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687479d-3e82-440c-a4bd-4da2bd240184 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-b687479d-3e82-440c-a4bd-4da2bd240184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:08:07.015292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:07.016546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:08:07.017606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:08:07.017606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:07.017606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:07.017606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 525/1032] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:08:07.022932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687479d-3e82-440c-a4bd-4da2bd240184 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:07.033078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce760220-da99-4f0c-9167-42338470c077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:07.033704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760220-da99-4f0c-9167-42338470c077 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:07.034530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760220-da99-4f0c-9167-42338470c077 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:07.035263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce760220-da99-4f0c-9167-42338470c077 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:07.036299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce760220-da99-4f0c-9167-42338470c077 None admin] RBAC: Adding query filter params (name=image) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:07.036741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce760220-da99-4f0c-9167-42338470c077 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:07.041400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce760220-da99-4f0c-9167-42338470c077 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:07.052903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 508/1033] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:07.132007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce760220-da99-4f0c-9167-42338470c077 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:07.141067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:07.202160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687479d-3e82-440c-a4bd-4da2bd240184 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:07.229397 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce760220-da99-4f0c-9167-42338470c077 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:07.282323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758a235-83a4-4db4-b19c-7585d6da13d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:07.283798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758a235-83a4-4db4-b19c-7585d6da13d3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:07.283798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758a235-83a4-4db4-b19c-7585d6da13d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:07.284169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758a235-83a4-4db4-b19c-7585d6da13d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:07.285263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758a235-83a4-4db4-b19c-7585d6da13d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:07.285748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758a235-83a4-4db4-b19c-7585d6da13d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:07.288425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d758a235-83a4-4db4-b19c-7585d6da13d3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:07.298879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64465e-be55-4d12-91bb-2422692cc856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:07.299911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64465e-be55-4d12-91bb-2422692cc856 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:07.300388 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64465e-be55-4d12-91bb-2422692cc856 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:07.300848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64465e-be55-4d12-91bb-2422692cc856 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:07.301978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64465e-be55-4d12-91bb-2422692cc856 None admin] RBAC: Adding query filter params (type=image) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:07.302502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64465e-be55-4d12-91bb-2422692cc856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:07.305327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64465e-be55-4d12-91bb-2422692cc856 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:07.305496 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 526/1034] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:07.311143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758a235-83a4-4db4-b19c-7585d6da13d3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:07.314777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 509/1035] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:07.325422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64465e-be55-4d12-91bb-2422692cc856 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:07.404819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:07.468053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758a235-83a4-4db4-b19c-7585d6da13d3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:07.490848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64465e-be55-4d12-91bb-2422692cc856 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:07.510943 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:07.529824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53783dc8-587f-42ae-9e77-aafce678320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:07.536220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53783dc8-587f-42ae-9e77-aafce678320e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:07.536220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53783dc8-587f-42ae-9e77-aafce678320e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:07.536220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53783dc8-587f-42ae-9e77-aafce678320e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:07.546009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53783dc8-587f-42ae-9e77-aafce678320e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:07.548470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53783dc8-587f-42ae-9e77-aafce678320e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:07.562720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558d84f-6e32-4b00-8c59-e95d4dadcfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:07.563709 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558d84f-6e32-4b00-8c59-e95d4dadcfae None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:07.564160 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558d84f-6e32-4b00-8c59-e95d4dadcfae None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:07.564588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558d84f-6e32-4b00-8c59-e95d4dadcfae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:07.565570 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558d84f-6e32-4b00-8c59-e95d4dadcfae None admin] RBAC: Adding query filter params (interface=public, service_id=579e20f1be214475af655248f3a0f430, region_id=RegionOne) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:07.566026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558d84f-6e32-4b00-8c59-e95d4dadcfae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:07.567750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558d84f-6e32-4b00-8c59-e95d4dadcfae None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-53783dc8-587f-42ae-9e77-aafce678320e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:08:07.568638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:07.570597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:07.571711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:07.571711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:07.571711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:08:07.571711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:08:07.571711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:08:07.571711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:08:07.571711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:08:07.571711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:08:07.571711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:08:07.571711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:07.575808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 527/1036] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Sep 23 01:08:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:08:07.579948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53783dc8-587f-42ae-9e77-aafce678320e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:07.593254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 510/1037] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Sep 23 01:08:07 2022] GET /identity/v3/endpoints?service_id=579e20f1be214475af655248f3a0f430&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:07.600776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558d84f-6e32-4b00-8c59-e95d4dadcfae None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:07.772492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53783dc8-587f-42ae-9e77-aafce678320e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:07.789232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:07.793078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558d84f-6e32-4b00-8c59-e95d4dadcfae None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:07.854320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc2ab4-a88b-4ecb-b77d-2070f93dec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:07.855162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc2ab4-a88b-4ecb-b77d-2070f93dec72 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:07.855732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc2ab4-a88b-4ecb-b77d-2070f93dec72 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:07.856095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc2ab4-a88b-4ecb-b77d-2070f93dec72 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:07.857559 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc2ab4-a88b-4ecb-b77d-2070f93dec72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:07.858096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc2ab4-a88b-4ecb-b77d-2070f93dec72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:07.861084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc2ab4-a88b-4ecb-b77d-2070f93dec72 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:07.878994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a71ce-3c33-450f-b451-0f5b78fa947c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:07.879988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a71ce-3c33-450f-b451-0f5b78fa947c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:07.880481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a71ce-3c33-450f-b451-0f5b78fa947c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:07.880965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a71ce-3c33-450f-b451-0f5b78fa947c None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:07.882135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a71ce-3c33-450f-b451-0f5b78fa947c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:07.884851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563a71ce-3c33-450f-b451-0f5b78fa947c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:07.900210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 511/1038] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Sep 23 01:08:07 2022] GET /identity/v3/services => generated 1957 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:08:07.900862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 528/1039] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Sep 23 01:08:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:07.910794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a71ce-3c33-450f-b451-0f5b78fa947c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:07.987913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:08.117014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a71ce-3c33-450f-b451-0f5b78fa947c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:08.183711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51254bb2-4fd4-4688-b859-b0780c21c5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:08.184673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51254bb2-4fd4-4688-b859-b0780c21c5ae None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:08.185142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51254bb2-4fd4-4688-b859-b0780c21c5ae None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:08.185646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51254bb2-4fd4-4688-b859-b0780c21c5ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:08.316655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51254bb2-4fd4-4688-b859-b0780c21c5ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:08.318418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51254bb2-4fd4-4688-b859-b0780c21c5ae None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:08.392609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:08.499615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 512/1040] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:08:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:08:08.506197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc2ab4-a88b-4ecb-b77d-2070f93dec72 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:08.640211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc2ab4-a88b-4ecb-b77d-2070f93dec72 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:08.695145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e58ab6-483a-4e8f-9db8-42654e83af92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:08.695948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e58ab6-483a-4e8f-9db8-42654e83af92 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:08.696350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e58ab6-483a-4e8f-9db8-42654e83af92 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:08.696621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e58ab6-483a-4e8f-9db8-42654e83af92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:08.710728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e58ab6-483a-4e8f-9db8-42654e83af92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:08.712291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e58ab6-483a-4e8f-9db8-42654e83af92 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-d0e58ab6-483a-4e8f-9db8-42654e83af92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:08:08.731202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:08.733415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:08:08.736394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:08:08.736394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:08.736394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:08.738246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 529/1041] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:08.740836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51254bb2-4fd4-4688-b859-b0780c21c5ae None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:08.899439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51254bb2-4fd4-4688-b859-b0780c21c5ae None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:08.961202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c22ff9-66b9-4690-a6f0-6658426641f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:08.961791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22ff9-66b9-4690-a6f0-6658426641f1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:08.962082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22ff9-66b9-4690-a6f0-6658426641f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:08.962359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c22ff9-66b9-4690-a6f0-6658426641f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:08.963626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c22ff9-66b9-4690-a6f0-6658426641f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:08.963955 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c22ff9-66b9-4690-a6f0-6658426641f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:08.966474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c22ff9-66b9-4690-a6f0-6658426641f1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:08.976802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:08.992785 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 513/1042] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:09.006679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e58ab6-483a-4e8f-9db8-42654e83af92 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:09.036009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:09.130901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f25b3-4ea6-4d09-bb0a-f45d0ab8b884 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:09.131515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f25b3-4ea6-4d09-bb0a-f45d0ab8b884 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:09.132022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f25b3-4ea6-4d09-bb0a-f45d0ab8b884 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:09.134894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 514/1043] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:09.144701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce6205-2f92-4e63-9d96-b8d37ed318d9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:09.145191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce6205-2f92-4e63-9d96-b8d37ed318d9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:09.145667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce6205-2f92-4e63-9d96-b8d37ed318d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:09.166273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e58ab6-483a-4e8f-9db8-42654e83af92 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:09.178231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-14ce6205-2f92-4e63-9d96-b8d37ed318d9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:09.181613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-14ce6205-2f92-4e63-9d96-b8d37ed318d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:09.220467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed7023-88e7-4ccc-856c-cace96dc8a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:09.221733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed7023-88e7-4ccc-856c-cace96dc8a9a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:09.222378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed7023-88e7-4ccc-856c-cace96dc8a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:09.223020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed7023-88e7-4ccc-856c-cace96dc8a9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:09.234138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed7023-88e7-4ccc-856c-cace96dc8a9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:09.236194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed7023-88e7-4ccc-856c-cace96dc8a9a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-9bed7023-88e7-4ccc-856c-cace96dc8a9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:08:09.252009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:09.253543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:09.254495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:08:09.254495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:08:09.254495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:08:09.254495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:08:09.254495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:08:09.254495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:08:09.254495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:08:09.254495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:09.254495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 530/1044] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:08:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:09.260139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed7023-88e7-4ccc-856c-cace96dc8a9a None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:09.294837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 515/1045] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:09.370080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:09.395973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed7023-88e7-4ccc-856c-cace96dc8a9a None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:09.458997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66619851-1522-44b9-a93b-82958ea109d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:09.459648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66619851-1522-44b9-a93b-82958ea109d5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:09.459941 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66619851-1522-44b9-a93b-82958ea109d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:09.460220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66619851-1522-44b9-a93b-82958ea109d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:09.460931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66619851-1522-44b9-a93b-82958ea109d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:09.461245 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66619851-1522-44b9-a93b-82958ea109d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:09.462699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66619851-1522-44b9-a93b-82958ea109d5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:09.482616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 531/1046] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:08:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:09.485061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce6205-2f92-4e63-9d96-b8d37ed318d9 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:09.579654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66619851-1522-44b9-a93b-82958ea109d5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:09.604467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce6205-2f92-4e63-9d96-b8d37ed318d9 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:09.637222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:09.647432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b66af-38e6-4733-9641-9df800895c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:09.648202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b66af-38e6-4733-9641-9df800895c50 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:09.652340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b66af-38e6-4733-9641-9df800895c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:09.652340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322b66af-38e6-4733-9641-9df800895c50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:09.652340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b66af-38e6-4733-9641-9df800895c50 None admin] RBAC: Adding query filter params (role.id=81d0f75541e3413d802620f47d89ed62, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=1c4da5b2130d4fa990adb72f0f6ec2fe) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:09.652340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b66af-38e6-4733-9641-9df800895c50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:09.652340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322b66af-38e6-4733-9641-9df800895c50 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:09.657611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 516/1047] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:08:09 2022] GET /identity/v3/role_assignments?user.id=1c4da5b2130d4fa990adb72f0f6ec2fe&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=81d0f75541e3413d802620f47d89ed62 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:09.704352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66619851-1522-44b9-a93b-82958ea109d5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:09.750164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90152f87-26bf-4b1c-92cd-b396510973c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:09.751233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90152f87-26bf-4b1c-92cd-b396510973c3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:09.751781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90152f87-26bf-4b1c-92cd-b396510973c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:09.752282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90152f87-26bf-4b1c-92cd-b396510973c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:09.761282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:09.767844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90152f87-26bf-4b1c-92cd-b396510973c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-90152f87-26bf-4b1c-92cd-b396510973c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:08:09.773407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:09.774454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:09.775335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:09.778150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 532/1048] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:08:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:08:09.788566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b66af-38e6-4733-9641-9df800895c50 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:09.935960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322b66af-38e6-4733-9641-9df800895c50 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:09.979709 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef399a2d-5414-44cd-85c9-d391ddd826c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:09.980321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef399a2d-5414-44cd-85c9-d391ddd826c1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:09.980689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef399a2d-5414-44cd-85c9-d391ddd826c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:09.980962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef399a2d-5414-44cd-85c9-d391ddd826c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:09.981628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef399a2d-5414-44cd-85c9-d391ddd826c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:09.981990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef399a2d-5414-44cd-85c9-d391ddd826c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:09.983360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef399a2d-5414-44cd-85c9-d391ddd826c1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:09.993194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 517/1049] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:08:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:09.999706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90152f87-26bf-4b1c-92cd-b396510973c3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:10.110892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90152f87-26bf-4b1c-92cd-b396510973c3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:10.148033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fb343-031a-41d2-b802-e34829ecb330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:10.148679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fb343-031a-41d2-b802-e34829ecb330 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:10.148961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fb343-031a-41d2-b802-e34829ecb330 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:10.149316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fb343-031a-41d2-b802-e34829ecb330 None admin] PATH_INFO: `/v3/users/9fe5b111887f4f3bae3dbda0bb07e79f` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:10.170907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fb343-031a-41d2-b802-e34829ecb330 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe5b111887f4f3bae3dbda0bb07e79f)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:10.175929 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fb343-031a-41d2-b802-e34829ecb330 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:10.179217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 533/1050] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:09 2022] GET /identity/v3/users/9fe5b111887f4f3bae3dbda0bb07e79f => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:10.184834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef399a2d-5414-44cd-85c9-d391ddd826c1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:10.251152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:10.301632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef399a2d-5414-44cd-85c9-d391ddd826c1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:10.339213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01da96-4a02-41ee-944f-e51a9f96b3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:10.339920 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01da96-4a02-41ee-944f-e51a9f96b3ee None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:10.340203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01da96-4a02-41ee-944f-e51a9f96b3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:10.340499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01da96-4a02-41ee-944f-e51a9f96b3ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:10.351461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7b0f7-bff2-4ebd-8ce3-e6c96e1dd626 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:10.351461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7b0f7-bff2-4ebd-8ce3-e6c96e1dd626 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:10.351461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7b0f7-bff2-4ebd-8ce3-e6c96e1dd626 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:10.354483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 534/1051] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:08:10.358327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd52577-5af0-45b8-be2c-fa539bff1a76 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:10.358686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd52577-5af0-45b8-be2c-fa539bff1a76 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:10.359574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd52577-5af0-45b8-be2c-fa539bff1a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:10.378744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9bd52577-5af0-45b8-be2c-fa539bff1a76 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:10.380022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9bd52577-5af0-45b8-be2c-fa539bff1a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:10.431817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01da96-4a02-41ee-944f-e51a9f96b3ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:10.433557 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01da96-4a02-41ee-944f-e51a9f96b3ee None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:10.451982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:10.482953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 535/1052] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:08:10.533486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:10.601362 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 518/1053] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:08:10 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:08:10.608199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd52577-5af0-45b8-be2c-fa539bff1a76 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:10.741428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd52577-5af0-45b8-be2c-fa539bff1a76 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:10.781560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01da96-4a02-41ee-944f-e51a9f96b3ee None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:10.784308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f146452-b1ea-4cbe-ac3b-d45700764fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:10.785292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f146452-b1ea-4cbe-ac3b-d45700764fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:10.785752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f146452-b1ea-4cbe-ac3b-d45700764fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:10.786032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f146452-b1ea-4cbe-ac3b-d45700764fa2 None admin] PATH_INFO: `/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:10.787221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f146452-b1ea-4cbe-ac3b-d45700764fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=8589ac3b9ead4fde8e2c2edc62907af5)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:10.788789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f146452-b1ea-4cbe-ac3b-d45700764fa2 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:10.791532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 536/1054] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:08:10 2022] GET /identity/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5 => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:10.797843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f146452-b1ea-4cbe-ac3b-d45700764fa2 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:10.898628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:10.911653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01da96-4a02-41ee-944f-e51a9f96b3ee None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:10.952519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c28a73-07a9-474d-9e0e-a7a5d872d84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:10.953649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c28a73-07a9-474d-9e0e-a7a5d872d84f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:10.953933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c28a73-07a9-474d-9e0e-a7a5d872d84f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:10.954213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c28a73-07a9-474d-9e0e-a7a5d872d84f None admin] PATH_INFO: `/v3/services/image` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:10.954812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c28a73-07a9-474d-9e0e-a7a5d872d84f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:10.956239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c28a73-07a9-474d-9e0e-a7a5d872d84f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:10.958103 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f146452-b1ea-4cbe-ac3b-d45700764fa2 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-04c28a73-07a9-474d-9e0e-a7a5d872d84f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:10.966395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:10.968503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:10.969725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 519/1055] 10.222.0.64 () {56 vars in 1105 bytes} [Fri Sep 23 01:08:10 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:10.972499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c28a73-07a9-474d-9e0e-a7a5d872d84f None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:10.993191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:11.005989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9e163-1dd0-4092-aef9-6609a14641ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:11.007602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9e163-1dd0-4092-aef9-6609a14641ca None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:11.007602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9e163-1dd0-4092-aef9-6609a14641ca None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:11.007602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9e163-1dd0-4092-aef9-6609a14641ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:11.008057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9e163-1dd0-4092-aef9-6609a14641ca None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=8589ac3b9ead4fde8e2c2edc62907af5, user.id=9fe5b111887f4f3bae3dbda0bb07e79f) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:11.008378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9e163-1dd0-4092-aef9-6609a14641ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:11.009827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9e163-1dd0-4092-aef9-6609a14641ca None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:11.017102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 537/1056] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:08:10 2022] GET /identity/v3/role_assignments?user.id=9fe5b111887f4f3bae3dbda0bb07e79f&scope.project.id=8589ac3b9ead4fde8e2c2edc62907af5&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:11.107835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c28a73-07a9-474d-9e0e-a7a5d872d84f None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:11.152245 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e68fd4-1644-4509-90da-bbdb958c209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:11.153332 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68fd4-1644-4509-90da-bbdb958c209c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:11.153789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68fd4-1644-4509-90da-bbdb958c209c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:11.154261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e68fd4-1644-4509-90da-bbdb958c209c None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:11.155119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e68fd4-1644-4509-90da-bbdb958c209c None admin] RBAC: Adding query filter params (name=image) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:11.155745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e68fd4-1644-4509-90da-bbdb958c209c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:11.157430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e68fd4-1644-4509-90da-bbdb958c209c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:11.164249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 520/1057] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:11.171819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9e163-1dd0-4092-aef9-6609a14641ca None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:11.302020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9e163-1dd0-4092-aef9-6609a14641ca None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:11.325508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:11.347042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7f458-10cd-45ec-ae6a-34441cacd266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:11.347703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f458-10cd-45ec-ae6a-34441cacd266 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:11.347979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f458-10cd-45ec-ae6a-34441cacd266 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:11.348256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7f458-10cd-45ec-ae6a-34441cacd266 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:11.348908 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7f458-10cd-45ec-ae6a-34441cacd266 None admin] RBAC: Adding query filter params (type=image) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:11.349404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7f458-10cd-45ec-ae6a-34441cacd266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:11.350604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7f458-10cd-45ec-ae6a-34441cacd266 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:11.362262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 538/1058] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:11.363584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e68fd4-1644-4509-90da-bbdb958c209c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:11.462846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:11.484833 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e68fd4-1644-4509-90da-bbdb958c209c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:11.526450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11cd08-c4f2-414b-8894-46c47ae8807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:11.527075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11cd08-c4f2-414b-8894-46c47ae8807a None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:11.527355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11cd08-c4f2-414b-8894-46c47ae8807a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:11.527669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11cd08-c4f2-414b-8894-46c47ae8807a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:11.528368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11cd08-c4f2-414b-8894-46c47ae8807a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:11.529746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11cd08-c4f2-414b-8894-46c47ae8807a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:11.635608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 521/1059] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:08:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:08:11.892549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4291cc-cd71-4f7f-83c9-99f882ff61f2 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:11.893083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4291cc-cd71-4f7f-83c9-99f882ff61f2 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:11.893083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4291cc-cd71-4f7f-83c9-99f882ff61f2 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:11.895591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 539/1060] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:08:11.903273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7b2b-f653-4e98-ab4a-51f4996c799f None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:11.903661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7b2b-f653-4e98-ab4a-51f4996c799f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:11.903941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b7b2b-f653-4e98-ab4a-51f4996c799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:11.927999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-cf9b7b2b-f653-4e98-ab4a-51f4996c799f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:11.929728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cf9b7b2b-f653-4e98-ab4a-51f4996c799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:12.042227 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 522/1061] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:12.316753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4291cc-cd71-4f7f-83c9-99f882ff61f2 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:12.478734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4291cc-cd71-4f7f-83c9-99f882ff61f2 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:12.536526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef037f0d-87bb-4797-88c5-a97db6dc47b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:12.537197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef037f0d-87bb-4797-88c5-a97db6dc47b3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:12.537480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef037f0d-87bb-4797-88c5-a97db6dc47b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:12.537977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef037f0d-87bb-4797-88c5-a97db6dc47b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:12.553031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef037f0d-87bb-4797-88c5-a97db6dc47b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-ef037f0d-87bb-4797-88c5-a97db6dc47b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:08:12.558453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:12.559531 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:12.564973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 540/1062] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Sep 23 01:08:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:12.570650 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7b2b-f653-4e98-ab4a-51f4996c799f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:12.656397 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:12.712629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b7b2b-f653-4e98-ab4a-51f4996c799f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:12.758238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b602118-1d58-482f-a4b4-0a41f7887f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:12.758872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b602118-1d58-482f-a4b4-0a41f7887f12 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:12.759154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b602118-1d58-482f-a4b4-0a41f7887f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:12.759432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b602118-1d58-482f-a4b4-0a41f7887f12 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:12.760157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b602118-1d58-482f-a4b4-0a41f7887f12 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:12.760464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b602118-1d58-482f-a4b4-0a41f7887f12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:12.761862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b602118-1d58-482f-a4b4-0a41f7887f12 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:12.776464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 523/1063] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Sep 23 01:08:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:12.778774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef037f0d-87bb-4797-88c5-a97db6dc47b3 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:12.837130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:12.904785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef037f0d-87bb-4797-88c5-a97db6dc47b3 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:12.946736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f86b0-73ba-466d-a787-af0dbeb1b455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:12.947382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f86b0-73ba-466d-a787-af0dbeb1b455 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:12.947689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f86b0-73ba-466d-a787-af0dbeb1b455 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:12.948040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f86b0-73ba-466d-a787-af0dbeb1b455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:12.960535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f86b0-73ba-466d-a787-af0dbeb1b455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:12.962030 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f86b0-73ba-466d-a787-af0dbeb1b455 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-e78f86b0-73ba-466d-a787-af0dbeb1b455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:08:12.981231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:12.983076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:08:12.985325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:08:12.985325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:12.985325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:12.988105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 541/1064] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:12.990595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b602118-1d58-482f-a4b4-0a41f7887f12 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:13.156995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b602118-1d58-482f-a4b4-0a41f7887f12 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:13.205295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:13.221211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ddbde-5562-43fe-974c-492460163151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:13.222251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ddbde-5562-43fe-974c-492460163151 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:13.222793 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ddbde-5562-43fe-974c-492460163151 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:13.223347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ddbde-5562-43fe-974c-492460163151 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:13.224552 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ddbde-5562-43fe-974c-492460163151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:13.225099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ddbde-5562-43fe-974c-492460163151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:13.227711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ddbde-5562-43fe-974c-492460163151 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:13.255005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 524/1065] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:13.257053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f86b0-73ba-466d-a787-af0dbeb1b455 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:13.398214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:13.412045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f86b0-73ba-466d-a787-af0dbeb1b455 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:13.451583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef1df1-5d8d-4099-adf4-15f8ec0c172a None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:13.451916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef1df1-5d8d-4099-adf4-15f8ec0c172a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:13.452199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef1df1-5d8d-4099-adf4-15f8ec0c172a None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:13.454151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 525/1066] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:08:13.455808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc24c98-583c-4491-9c78-49ca2d9886b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:13.457077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc24c98-583c-4491-9c78-49ca2d9886b0 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:13.457764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc24c98-583c-4491-9c78-49ca2d9886b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:13.458459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc24c98-583c-4491-9c78-49ca2d9886b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:13.460107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a7bfc-9763-4478-9042-3bc30055e099 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:13.460413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a7bfc-9763-4478-9042-3bc30055e099 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:13.460660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a7bfc-9763-4478-9042-3bc30055e099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:13.475616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc24c98-583c-4491-9c78-49ca2d9886b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:13.476804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc24c98-583c-4491-9c78-49ca2d9886b0 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:13.480131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-130a7bfc-9763-4478-9042-3bc30055e099 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:13.481423 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-130a7bfc-9763-4478-9042-3bc30055e099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-fbc24c98-583c-4491-9c78-49ca2d9886b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:08:13.495231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:13.496312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:13.502660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 542/1067] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Sep 23 01:08:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:13.504823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc24c98-583c-4491-9c78-49ca2d9886b0 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:13.589255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 526/1068] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:13.696521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc24c98-583c-4491-9c78-49ca2d9886b0 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:13.762374 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739997ff-bfa2-4854-ab35-a9e5cee1271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:13.763510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739997ff-bfa2-4854-ab35-a9e5cee1271b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:13.764087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739997ff-bfa2-4854-ab35-a9e5cee1271b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:13.764575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739997ff-bfa2-4854-ab35-a9e5cee1271b None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:13.766269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739997ff-bfa2-4854-ab35-a9e5cee1271b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:13.766822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739997ff-bfa2-4854-ab35-a9e5cee1271b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:13.769558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739997ff-bfa2-4854-ab35-a9e5cee1271b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:13.807382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 543/1069] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Sep 23 01:08:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:13.816876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a7bfc-9763-4478-9042-3bc30055e099 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:13.976854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739997ff-bfa2-4854-ab35-a9e5cee1271b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:13.978901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a7bfc-9763-4478-9042-3bc30055e099 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:14.009026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:14.009941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:14.033435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e96c5-6fc3-400f-a51f-f084dde3f0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:14.034566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e96c5-6fc3-400f-a51f-f084dde3f0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:14.034954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e96c5-6fc3-400f-a51f-f084dde3f0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:14.035346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e96c5-6fc3-400f-a51f-f084dde3f0c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:14.047264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e96c5-6fc3-400f-a51f-f084dde3f0c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:14.048792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e96c5-6fc3-400f-a51f-f084dde3f0c0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-db7e96c5-6fc3-400f-a51f-f084dde3f0c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:08:14.066687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:14.068018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:08:14.069172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:08:14.069172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:14.069172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:14.069172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 527/1070] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:08:14.077683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e96c5-6fc3-400f-a51f-f084dde3f0c0 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:14.149431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739997ff-bfa2-4854-ab35-a9e5cee1271b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:14.209605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86330d8a-ac09-4138-80ff-ddff9df66db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:14.210389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86330d8a-ac09-4138-80ff-ddff9df66db4 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:14.210745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86330d8a-ac09-4138-80ff-ddff9df66db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:14.211082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86330d8a-ac09-4138-80ff-ddff9df66db4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:14.224444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86330d8a-ac09-4138-80ff-ddff9df66db4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-86330d8a-ac09-4138-80ff-ddff9df66db4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:08:14.229713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:14.230726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:14.231595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 544/1071] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:08:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:14.232663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0004e5-d170-49a6-a007-138c53330e4c None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:14.233030 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0004e5-d170-49a6-a007-138c53330e4c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:14.233164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0004e5-d170-49a6-a007-138c53330e4c None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:14.235969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 545/1072] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:14.239387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0004e5-d170-49a6-a007-138c53330e4c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:14.242059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e96c5-6fc3-400f-a51f-f084dde3f0c0 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:14.286466 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b717846-a8e7-48cc-b5a2-e00dfc6f7722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:14.287096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b717846-a8e7-48cc-b5a2-e00dfc6f7722 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:14.287412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b717846-a8e7-48cc-b5a2-e00dfc6f7722 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:14.287730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b717846-a8e7-48cc-b5a2-e00dfc6f7722 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:14.288392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b717846-a8e7-48cc-b5a2-e00dfc6f7722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:14.288709 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b717846-a8e7-48cc-b5a2-e00dfc6f7722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:14.290109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b717846-a8e7-48cc-b5a2-e00dfc6f7722 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:14.305673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 528/1073] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:14.306951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a0eb8-3360-4159-97d1-a8291fa9aaa2 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:14.308150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a0eb8-3360-4159-97d1-a8291fa9aaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:14.308150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a0eb8-3360-4159-97d1-a8291fa9aaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:14.340697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-394a0eb8-3360-4159-97d1-a8291fa9aaa2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:14.342513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-394a0eb8-3360-4159-97d1-a8291fa9aaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:14.367603 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:14.380137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:14.416466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0004e5-d170-49a6-a007-138c53330e4c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:14.455206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 529/1074] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:14.456608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a0eb8-3360-4159-97d1-a8291fa9aaa2 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:14.476042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6f152-6891-46d3-b68d-999193d2d376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:14.476863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f152-6891-46d3-b68d-999193d2d376 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:14.477415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f152-6891-46d3-b68d-999193d2d376 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:14.477669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6f152-6891-46d3-b68d-999193d2d376 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:14.478386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6f152-6891-46d3-b68d-999193d2d376 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:14.478719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6f152-6891-46d3-b68d-999193d2d376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:14.481328 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6f152-6891-46d3-b68d-999193d2d376 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:14.495635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 546/1075] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:08:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:14.500491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6f152-6891-46d3-b68d-999193d2d376 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:14.594766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a0eb8-3360-4159-97d1-a8291fa9aaa2 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:14.639316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029a6c5-8975-4220-bd1d-2afc6412da83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:14.640308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029a6c5-8975-4220-bd1d-2afc6412da83 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:14.640590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029a6c5-8975-4220-bd1d-2afc6412da83 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:14.640864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029a6c5-8975-4220-bd1d-2afc6412da83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:14.643125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6f152-6891-46d3-b68d-999193d2d376 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:14.649546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6029a6c5-8975-4220-bd1d-2afc6412da83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:14.651290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6029a6c5-8975-4220-bd1d-2afc6412da83 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-6029a6c5-8975-4220-bd1d-2afc6412da83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:08:14.667298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:14.673342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:14.676914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:08:14.676914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:08:14.676914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:08:14.676914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:08:14.676914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:08:14.676914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:08:14.676914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:08:14.676914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:14.676914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 530/1076] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:14.678813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029a6c5-8975-4220-bd1d-2afc6412da83 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:14.698708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60e579-aaa6-4433-8f43-1871e56a7e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:14.699368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60e579-aaa6-4433-8f43-1871e56a7e1e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:14.699702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60e579-aaa6-4433-8f43-1871e56a7e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:14.700018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60e579-aaa6-4433-8f43-1871e56a7e1e None admin] PATH_INFO: `/v3/users/9fe5b111887f4f3bae3dbda0bb07e79f` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:14.701416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60e579-aaa6-4433-8f43-1871e56a7e1e None admin] RBAC: Authorizing `identity:get_user(user_id=9fe5b111887f4f3bae3dbda0bb07e79f)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:14.702148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:14.703867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60e579-aaa6-4433-8f43-1871e56a7e1e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:14.705965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 547/1077] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:14 2022] GET /identity/v3/users/9fe5b111887f4f3bae3dbda0bb07e79f => generated 285 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:14.711335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60e579-aaa6-4433-8f43-1871e56a7e1e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:14.714431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:14.807030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6029a6c5-8975-4220-bd1d-2afc6412da83 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:14.843759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60e579-aaa6-4433-8f43-1871e56a7e1e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:14.849467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c50aa8-e9e5-4b7c-b71a-3a6f17e5e904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:14.850084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c50aa8-e9e5-4b7c-b71a-3a6f17e5e904 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:14.850358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c50aa8-e9e5-4b7c-b71a-3a6f17e5e904 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:14.850628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c50aa8-e9e5-4b7c-b71a-3a6f17e5e904 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:14.851330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c50aa8-e9e5-4b7c-b71a-3a6f17e5e904 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:14.851670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c50aa8-e9e5-4b7c-b71a-3a6f17e5e904 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:14.853190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c50aa8-e9e5-4b7c-b71a-3a6f17e5e904 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:14.868991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 531/1078] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:08:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:14.869778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c50aa8-e9e5-4b7c-b71a-3a6f17e5e904 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:14.887298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416a04e-2d64-4c64-8b19-1e16b0240d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:14.887979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416a04e-2d64-4c64-8b19-1e16b0240d4c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:14.888262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416a04e-2d64-4c64-8b19-1e16b0240d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:14.888537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416a04e-2d64-4c64-8b19-1e16b0240d4c None admin] PATH_INFO: `/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:14.889709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416a04e-2d64-4c64-8b19-1e16b0240d4c None admin] RBAC: Authorizing `identity:get_project(project_id=8589ac3b9ead4fde8e2c2edc62907af5)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:14.891706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416a04e-2d64-4c64-8b19-1e16b0240d4c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:14.893452 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 548/1079] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:08:14 2022] GET /identity/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5 => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:14.894217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416a04e-2d64-4c64-8b19-1e16b0240d4c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:14.964263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:14.975187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:14.986706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c50aa8-e9e5-4b7c-b71a-3a6f17e5e904 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:15.008097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416a04e-2d64-4c64-8b19-1e16b0240d4c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:15.023372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51856a8-02cb-470e-8452-39f05df1d658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:15.024020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51856a8-02cb-470e-8452-39f05df1d658 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:15.024286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51856a8-02cb-470e-8452-39f05df1d658 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:15.024554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51856a8-02cb-470e-8452-39f05df1d658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:15.046177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a2991-eeae-4b6f-9aa0-23f09b96395c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:15.046857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a2991-eeae-4b6f-9aa0-23f09b96395c None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:15.047140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a2991-eeae-4b6f-9aa0-23f09b96395c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:15.047417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a2991-eeae-4b6f-9aa0-23f09b96395c None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/1c4da5b2130d4fa990adb72f0f6ec2fe/roles/81d0f75541e3413d802620f47d89ed62` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:15.049709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a2991-eeae-4b6f-9aa0-23f09b96395c None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=1c4da5b2130d4fa990adb72f0f6ec2fe, role_id=81d0f75541e3413d802620f47d89ed62)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:15.051154 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a2991-eeae-4b6f-9aa0-23f09b96395c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:15.095848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51856a8-02cb-470e-8452-39f05df1d658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:15.097237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51856a8-02cb-470e-8452-39f05df1d658 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:15.174122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:15.190721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 549/1080] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:08:14 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/1c4da5b2130d4fa990adb72f0f6ec2fe/roles/81d0f75541e3413d802620f47d89ed62 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:08:15.191542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a2991-eeae-4b6f-9aa0-23f09b96395c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:15.196604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 532/1081] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:08:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:08:15.203268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51856a8-02cb-470e-8452-39f05df1d658 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:15.323731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a2991-eeae-4b6f-9aa0-23f09b96395c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:15.329654 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51856a8-02cb-470e-8452-39f05df1d658 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:15.356907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:15.363507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b00b89-bdc4-4863-adc0-cd8b446dd4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:15.364218 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b00b89-bdc4-4863-adc0-cd8b446dd4f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:15.364607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b00b89-bdc4-4863-adc0-cd8b446dd4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:15.364790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b00b89-bdc4-4863-adc0-cd8b446dd4f3 None admin] PATH_INFO: `/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5/users/9fe5b111887f4f3bae3dbda0bb07e79f/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:15.367756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b00b89-bdc4-4863-adc0-cd8b446dd4f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8589ac3b9ead4fde8e2c2edc62907af5, user_id=9fe5b111887f4f3bae3dbda0bb07e79f, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:15.368333 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b00b89-bdc4-4863-adc0-cd8b446dd4f3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:15.374089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198e41a-659f-4792-a3fc-00c5cc3d5e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:15.374716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198e41a-659f-4792-a3fc-00c5cc3d5e1a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:15.375002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198e41a-659f-4792-a3fc-00c5cc3d5e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:15.375282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198e41a-659f-4792-a3fc-00c5cc3d5e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:15.388149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198e41a-659f-4792-a3fc-00c5cc3d5e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:15.389634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198e41a-659f-4792-a3fc-00c5cc3d5e1a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-9198e41a-659f-4792-a3fc-00c5cc3d5e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:08:15.406269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:15.407700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:08:15.408609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:08:15.408609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:15.408609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:15.408609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 533/1082] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:08:15.413555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198e41a-659f-4792-a3fc-00c5cc3d5e1a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:15.450832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:15.467885 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 550/1083] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:08:15 2022] PUT /identity/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5/users/9fe5b111887f4f3bae3dbda0bb07e79f/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:08:15.557084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198e41a-659f-4792-a3fc-00c5cc3d5e1a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:15.618756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b0f9f-f448-463a-864e-d4ccd0104cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:15.619928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b0f9f-f448-463a-864e-d4ccd0104cff None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:15.620178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b0f9f-f448-463a-864e-d4ccd0104cff None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:15.620636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b0f9f-f448-463a-864e-d4ccd0104cff None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:15.621530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b0f9f-f448-463a-864e-d4ccd0104cff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:15.621962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b0f9f-f448-463a-864e-d4ccd0104cff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:15.624066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b0f9f-f448-463a-864e-d4ccd0104cff None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:15.642039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 534/1084] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:17.438896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae32d59-9070-4747-ac17-ef53f7ac9379 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:17.439469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae32d59-9070-4747-ac17-ef53f7ac9379 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:17.439469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae32d59-9070-4747-ac17-ef53f7ac9379 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:17.440871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 551/1085] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:17.446626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e41ab-38e9-40ee-9b80-e61a879d27ed None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:17.447077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e41ab-38e9-40ee-9b80-e61a879d27ed None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:17.447077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e41ab-38e9-40ee-9b80-e61a879d27ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:17.466320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-380e41ab-38e9-40ee-9b80-e61a879d27ed None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:17.467561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-380e41ab-38e9-40ee-9b80-e61a879d27ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:17.542690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 535/1086] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:17.706594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493ee31-2e19-4982-a84a-06a4fee9f24d None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:17.706800 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493ee31-2e19-4982-a84a-06a4fee9f24d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:17.707050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493ee31-2e19-4982-a84a-06a4fee9f24d None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:17.708482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 552/1087] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:17.714149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a391392-111e-4f4e-a530-be6f36e823e1 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:17.714442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a391392-111e-4f4e-a530-be6f36e823e1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:17.714720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a391392-111e-4f4e-a530-be6f36e823e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:17.733416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-9a391392-111e-4f4e-a530-be6f36e823e1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:17.734629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9a391392-111e-4f4e-a530-be6f36e823e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:17.751092 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:17.808404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 536/1088] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:17.817236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493ee31-2e19-4982-a84a-06a4fee9f24d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:17.941623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493ee31-2e19-4982-a84a-06a4fee9f24d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:17.981662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871c047-4956-4c58-b45d-c8e401f29a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:17.982282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871c047-4956-4c58-b45d-c8e401f29a1d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:17.982566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871c047-4956-4c58-b45d-c8e401f29a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:17.982835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871c047-4956-4c58-b45d-c8e401f29a1d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:17.994901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9871c047-4956-4c58-b45d-c8e401f29a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-9871c047-4956-4c58-b45d-c8e401f29a1d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:08:17.999893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:18.000891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:18.001828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 553/1089] 10.222.0.64 () {56 vars in 1120 bytes} [Fri Sep 23 01:08:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:18.007524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a391392-111e-4f4e-a530-be6f36e823e1 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:18.084313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871c047-4956-4c58-b45d-c8e401f29a1d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:18.129691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a391392-111e-4f4e-a530-be6f36e823e1 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:18.160879 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:18.179148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d6487-4453-4032-a721-9de7db652877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:18.179797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d6487-4453-4032-a721-9de7db652877 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:18.180070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d6487-4453-4032-a721-9de7db652877 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:18.180346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d6487-4453-4032-a721-9de7db652877 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:18.181039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d6487-4453-4032-a721-9de7db652877 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:18.181554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d6487-4453-4032-a721-9de7db652877 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:18.184409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d6487-4453-4032-a721-9de7db652877 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:18.200112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 537/1090] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Sep 23 01:08:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:18.206289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d6487-4453-4032-a721-9de7db652877 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:18.222048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:18.222258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871c047-4956-4c58-b45d-c8e401f29a1d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:18.260713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f166a6-a453-4d9c-845a-4d3335c2fc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:18.261352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f166a6-a453-4d9c-845a-4d3335c2fc18 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:18.261632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f166a6-a453-4d9c-845a-4d3335c2fc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:18.261988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f166a6-a453-4d9c-845a-4d3335c2fc18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:18.282681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f166a6-a453-4d9c-845a-4d3335c2fc18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-66f166a6-a453-4d9c-845a-4d3335c2fc18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:08:18.288255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:18.289766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:18.290994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 554/1091] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:08:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:08:18.297426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f166a6-a453-4d9c-845a-4d3335c2fc18 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:18.328102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d6487-4453-4032-a721-9de7db652877 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:18.374611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f7b21-14ac-4cbf-986f-8775dcb7d202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:18.375272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f7b21-14ac-4cbf-986f-8775dcb7d202 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:18.375806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f7b21-14ac-4cbf-986f-8775dcb7d202 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:18.376150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f7b21-14ac-4cbf-986f-8775dcb7d202 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:18.387444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f7b21-14ac-4cbf-986f-8775dcb7d202 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:18.388953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f7b21-14ac-4cbf-986f-8775dcb7d202 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-e85f7b21-14ac-4cbf-986f-8775dcb7d202 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:08:18.404754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:18.410281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:08:18.411218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:08:18.411218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:18.411218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:18.411218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 538/1092] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:08:18.411764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f166a6-a453-4d9c-845a-4d3335c2fc18 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:18.412309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f7b21-14ac-4cbf-986f-8775dcb7d202 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:18.424574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:18.454694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7815de-45a7-403b-9716-1758a2fe21bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:18.455781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7815de-45a7-403b-9716-1758a2fe21bc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:18.455781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7815de-45a7-403b-9716-1758a2fe21bc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:18.456099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7815de-45a7-403b-9716-1758a2fe21bc None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:18.456770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7815de-45a7-403b-9716-1758a2fe21bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:18.457085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7815de-45a7-403b-9716-1758a2fe21bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:18.458585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7815de-45a7-403b-9716-1758a2fe21bc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:18.470770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 555/1093] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:08:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:18.477163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7815de-45a7-403b-9716-1758a2fe21bc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:18.489088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:18.532017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f7b21-14ac-4cbf-986f-8775dcb7d202 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:18.567660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a4911-c9cb-4981-bd5f-1bba34a52720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:18.568474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4911-c9cb-4981-bd5f-1bba34a52720 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:18.568891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4911-c9cb-4981-bd5f-1bba34a52720 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:18.569311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4911-c9cb-4981-bd5f-1bba34a52720 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:18.570380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a4911-c9cb-4981-bd5f-1bba34a52720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:18.570777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a4911-c9cb-4981-bd5f-1bba34a52720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:18.572532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a4911-c9cb-4981-bd5f-1bba34a52720 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:18.586712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 539/1094] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:18.588208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7815de-45a7-403b-9716-1758a2fe21bc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:18.594522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a4911-c9cb-4981-bd5f-1bba34a52720 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:18.633358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7efd2-c232-4de5-a780-f88b27312736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:18.634012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7efd2-c232-4de5-a780-f88b27312736 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:18.634297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7efd2-c232-4de5-a780-f88b27312736 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:18.634631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7efd2-c232-4de5-a780-f88b27312736 None admin] PATH_INFO: `/v3/users/9fe5b111887f4f3bae3dbda0bb07e79f` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:18.635904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7efd2-c232-4de5-a780-f88b27312736 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe5b111887f4f3bae3dbda0bb07e79f)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:18.637406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7efd2-c232-4de5-a780-f88b27312736 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:18.639149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 556/1095] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:18 2022] GET /identity/v3/users/9fe5b111887f4f3bae3dbda0bb07e79f => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:18.645923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7efd2-c232-4de5-a780-f88b27312736 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:18.703480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:18.722593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a4911-c9cb-4981-bd5f-1bba34a52720 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:18.752252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:18.766147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034f2f1-430a-4660-8598-0d0651803525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:18.766902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034f2f1-430a-4660-8598-0d0651803525 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:18.767482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034f2f1-430a-4660-8598-0d0651803525 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:18.767482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034f2f1-430a-4660-8598-0d0651803525 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:18.775888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034f2f1-430a-4660-8598-0d0651803525 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:18.777309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034f2f1-430a-4660-8598-0d0651803525 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:18.778938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7efd2-c232-4de5-a780-f88b27312736 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-5034f2f1-430a-4660-8598-0d0651803525 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:08:18.791444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:18.792749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:18.793942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 540/1096] 10.222.0.64 () {56 vars in 1093 bytes} [Fri Sep 23 01:08:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:08:18.799075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034f2f1-430a-4660-8598-0d0651803525 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:18.818137 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b84838-c45c-498c-9d23-316c2624e994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:18.818762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b84838-c45c-498c-9d23-316c2624e994 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:18.819047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b84838-c45c-498c-9d23-316c2624e994 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:18.819321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b84838-c45c-498c-9d23-316c2624e994 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:18.896335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b84838-c45c-498c-9d23-316c2624e994 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:18.897868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b84838-c45c-498c-9d23-316c2624e994 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:18.918675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034f2f1-430a-4660-8598-0d0651803525 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:18.956085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c74326-b5a1-4924-b728-9f6fd43fd6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:18.957037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74326-b5a1-4924-b728-9f6fd43fd6bf None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:18.957037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74326-b5a1-4924-b728-9f6fd43fd6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:18.957292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74326-b5a1-4924-b728-9f6fd43fd6bf None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:18.958045 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c74326-b5a1-4924-b728-9f6fd43fd6bf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:18.958359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c74326-b5a1-4924-b728-9f6fd43fd6bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:18.959850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c74326-b5a1-4924-b728-9f6fd43fd6bf None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:18.969675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:18.984541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 541/1097] 10.222.0.64 () {56 vars in 1133 bytes} [Fri Sep 23 01:08:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:18.990736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c74326-b5a1-4924-b728-9f6fd43fd6bf None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:19.052927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 557/1098] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:08:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 01:08:19.054135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b84838-c45c-498c-9d23-316c2624e994 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:19.060350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:19.135301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c74326-b5a1-4924-b728-9f6fd43fd6bf None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:19.171874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b84838-c45c-498c-9d23-316c2624e994 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:19.180319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e40c2-dcb1-4dd3-851f-aef0fa936ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:19.181396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e40c2-dcb1-4dd3-851f-aef0fa936ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:19.181602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e40c2-dcb1-4dd3-851f-aef0fa936ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:19.181989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e40c2-dcb1-4dd3-851f-aef0fa936ef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:19.222667 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3444e8-030a-4d18-a8ef-8b6a902ee900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:19.223336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3444e8-030a-4d18-a8ef-8b6a902ee900 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:19.223649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3444e8-030a-4d18-a8ef-8b6a902ee900 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:19.223920 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3444e8-030a-4d18-a8ef-8b6a902ee900 None admin] PATH_INFO: `/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:19.225128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3444e8-030a-4d18-a8ef-8b6a902ee900 None admin] RBAC: Authorizing `identity:get_project(project_id=8589ac3b9ead4fde8e2c2edc62907af5)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:19.226587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3444e8-030a-4d18-a8ef-8b6a902ee900 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:19.228350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 558/1099] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:08:19 2022] GET /identity/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:19.235406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3444e8-030a-4d18-a8ef-8b6a902ee900 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:19.249984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:19.262987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e40c2-dcb1-4dd3-851f-aef0fa936ef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:19.264755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e40c2-dcb1-4dd3-851f-aef0fa936ef0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:19.314177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:19.350936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3444e8-030a-4d18-a8ef-8b6a902ee900 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:19.378725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 542/1100] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:08:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 01:08:19.384916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e40c2-dcb1-4dd3-851f-aef0fa936ef0 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:19.400835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d91b48-b3c3-43bb-971c-d8b13f4da264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:19.401455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d91b48-b3c3-43bb-971c-d8b13f4da264 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:19.401732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d91b48-b3c3-43bb-971c-d8b13f4da264 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:19.402065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d91b48-b3c3-43bb-971c-d8b13f4da264 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:19.402745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d91b48-b3c3-43bb-971c-d8b13f4da264 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=8589ac3b9ead4fde8e2c2edc62907af5, user.id=9fe5b111887f4f3bae3dbda0bb07e79f) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:19.403052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d91b48-b3c3-43bb-971c-d8b13f4da264 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:19.404544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d91b48-b3c3-43bb-971c-d8b13f4da264 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:19.410709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 559/1101] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:08:19 2022] GET /identity/v3/role_assignments?user.id=9fe5b111887f4f3bae3dbda0bb07e79f&scope.project.id=8589ac3b9ead4fde8e2c2edc62907af5&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:19.515703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e40c2-dcb1-4dd3-851f-aef0fa936ef0 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:19.557323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbd61d-df34-419a-8540-8d5fb2b9b44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:19.557984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbd61d-df34-419a-8540-8d5fb2b9b44a None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:19.558263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbd61d-df34-419a-8540-8d5fb2b9b44a None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:19.558537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbd61d-df34-419a-8540-8d5fb2b9b44a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:19.573070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbd61d-df34-419a-8540-8d5fb2b9b44a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:19.574489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fbd61d-df34-419a-8540-8d5fb2b9b44a None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:19.576735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-e9fbd61d-df34-419a-8540-8d5fb2b9b44a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:08:19.592384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:19.593478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:08:19.594383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:08:19.594383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:08:19.594383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:19.594383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 543/1102] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:08:19.599987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d91b48-b3c3-43bb-971c-d8b13f4da264 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:19.687901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:19.727640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d91b48-b3c3-43bb-971c-d8b13f4da264 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:19.765902 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d203f-116b-4b70-8682-cf0aedeb3489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:19.766515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d203f-116b-4b70-8682-cf0aedeb3489 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:19.767061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d203f-116b-4b70-8682-cf0aedeb3489 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:19.767061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d203f-116b-4b70-8682-cf0aedeb3489 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:19.767845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d203f-116b-4b70-8682-cf0aedeb3489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:19.768051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d203f-116b-4b70-8682-cf0aedeb3489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:19.769554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d203f-116b-4b70-8682-cf0aedeb3489 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:19.788137 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 560/1103] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:19.789669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbd61d-df34-419a-8540-8d5fb2b9b44a None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:19.904241 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbd61d-df34-419a-8540-8d5fb2b9b44a None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:19.942139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557918df-e561-4180-8119-2e259eb2f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:19.942788 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557918df-e561-4180-8119-2e259eb2f912 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:19.943063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557918df-e561-4180-8119-2e259eb2f912 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:19.943332 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557918df-e561-4180-8119-2e259eb2f912 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:19.951707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557918df-e561-4180-8119-2e259eb2f912 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:19.953251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557918df-e561-4180-8119-2e259eb2f912 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-557918df-e561-4180-8119-2e259eb2f912 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:08:19.967425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:19.969110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:19.970065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:08:19.970065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:08:19.970065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:08:19.970065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:08:19.970065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:08:19.970065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:08:19.970065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:08:19.970065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:19.970065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 544/1104] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:08:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:19.975754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d203f-116b-4b70-8682-cf0aedeb3489 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:20.077230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d203f-116b-4b70-8682-cf0aedeb3489 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:20.107860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:20.113392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7498f3-99ed-4cc2-9983-9f84b33a7fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:20.114007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7498f3-99ed-4cc2-9983-9f84b33a7fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:20.114287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7498f3-99ed-4cc2-9983-9f84b33a7fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:20.114551 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7498f3-99ed-4cc2-9983-9f84b33a7fe0 None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:20.115251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7498f3-99ed-4cc2-9983-9f84b33a7fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:20.115560 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7498f3-99ed-4cc2-9983-9f84b33a7fe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:20.116999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7498f3-99ed-4cc2-9983-9f84b33a7fe0 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:20.129448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 561/1105] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:08:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:20.135622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557918df-e561-4180-8119-2e259eb2f912 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:20.195629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:20.242214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557918df-e561-4180-8119-2e259eb2f912 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:20.276841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93701492-5fb5-41d4-8b20-d3c901133795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:20.277452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93701492-5fb5-41d4-8b20-d3c901133795 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:20.277731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93701492-5fb5-41d4-8b20-d3c901133795 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:20.278000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93701492-5fb5-41d4-8b20-d3c901133795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:20.278731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93701492-5fb5-41d4-8b20-d3c901133795 None admin] RBAC: Adding query filter params (role.id=81d0f75541e3413d802620f47d89ed62, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=1c4da5b2130d4fa990adb72f0f6ec2fe) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:20.279037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93701492-5fb5-41d4-8b20-d3c901133795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:20.280503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93701492-5fb5-41d4-8b20-d3c901133795 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:20.287305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 545/1106] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:08:20 2022] GET /identity/v3/role_assignments?user.id=1c4da5b2130d4fa990adb72f0f6ec2fe&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=81d0f75541e3413d802620f47d89ed62 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:21.670643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb88c01-f7d5-4b9e-ad8d-db56ff2125ab None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:21.671206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb88c01-f7d5-4b9e-ad8d-db56ff2125ab None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:21.671304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb88c01-f7d5-4b9e-ad8d-db56ff2125ab None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:21.672713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 562/1107] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:21.678094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d603-51d4-4691-bf38-3d8019922870 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:21.678526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d603-51d4-4691-bf38-3d8019922870 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:21.678629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40d603-51d4-4691-bf38-3d8019922870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:21.697796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-4d40d603-51d4-4691-bf38-3d8019922870 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:21.699002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4d40d603-51d4-4691-bf38-3d8019922870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:21.796296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 546/1108] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:22.087982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb88c01-f7d5-4b9e-ad8d-db56ff2125ab None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:22.211109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb88c01-f7d5-4b9e-ad8d-db56ff2125ab None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:22.245639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f44792-197a-4c7c-8d1c-deb85ba26072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:22.246276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44792-197a-4c7c-8d1c-deb85ba26072 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:22.246640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44792-197a-4c7c-8d1c-deb85ba26072 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:22.246835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44792-197a-4c7c-8d1c-deb85ba26072 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:22.248005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f44792-197a-4c7c-8d1c-deb85ba26072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:22.249490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f44792-197a-4c7c-8d1c-deb85ba26072 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:22.251111 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 563/1109] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:22.256906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40d603-51d4-4691-bf38-3d8019922870 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:22.306943 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:22.355237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40d603-51d4-4691-bf38-3d8019922870 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:22.391114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb37e6f-f6e0-4564-91e8-4e295e349a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:22.391744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb37e6f-f6e0-4564-91e8-4e295e349a26 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:22.392021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb37e6f-f6e0-4564-91e8-4e295e349a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:22.392291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb37e6f-f6e0-4564-91e8-4e295e349a26 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:22.392956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb37e6f-f6e0-4564-91e8-4e295e349a26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:22.394388 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb37e6f-f6e0-4564-91e8-4e295e349a26 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:22.511137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 547/1110] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:08:22 2022] POST /identity/v3/users => generated 286 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:08:24.809049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec453cb6-8081-42f4-a659-dfd5475c7d50 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:24.809705 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec453cb6-8081-42f4-a659-dfd5475c7d50 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:24.809705 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec453cb6-8081-42f4-a659-dfd5475c7d50 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:24.811529 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 564/1111] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:24.818589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321291f-aa4a-4f08-b660-69718a4dca0d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:24.819133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321291f-aa4a-4f08-b660-69718a4dca0d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:24.819258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321291f-aa4a-4f08-b660-69718a4dca0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:24.838278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f321291f-aa4a-4f08-b660-69718a4dca0d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:24.839493 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f321291f-aa4a-4f08-b660-69718a4dca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:24.913940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 548/1112] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:08:25.181057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec453cb6-8081-42f4-a659-dfd5475c7d50 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:25.246414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:25.292100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec453cb6-8081-42f4-a659-dfd5475c7d50 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:25.393950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d1b80-4ecd-4441-bca4-9a2d6bf06191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:25.394591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1b80-4ecd-4441-bca4-9a2d6bf06191 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:25.394870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1b80-4ecd-4441-bca4-9a2d6bf06191 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:25.395171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d1b80-4ecd-4441-bca4-9a2d6bf06191 None admin] PATH_INFO: `/v3/roles/b9c95d84eaca4444b2c7b8b5eed61a59` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:25.396621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d1b80-4ecd-4441-bca4-9a2d6bf06191 None admin] RBAC: Authorizing `identity:get_role(role_id=b9c95d84eaca4444b2c7b8b5eed61a59)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:25.397989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d1b80-4ecd-4441-bca4-9a2d6bf06191 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:25.399375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 565/1113] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:25 2022] GET /identity/v3/roles/b9c95d84eaca4444b2c7b8b5eed61a59 => generated 223 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:25.406845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321291f-aa4a-4f08-b660-69718a4dca0d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:25.442122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:25.524478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321291f-aa4a-4f08-b660-69718a4dca0d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:25.562940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2f1f4-6ed4-4262-bc3b-9d7a7640813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:25.563597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f1f4-6ed4-4262-bc3b-9d7a7640813e None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:25.563921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f1f4-6ed4-4262-bc3b-9d7a7640813e None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:25.564148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f1f4-6ed4-4262-bc3b-9d7a7640813e None admin] PATH_INFO: `/v3/users/d30edb8c976e4642a7b873e1ad9ced36` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:25.579139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2f1f4-6ed4-4262-bc3b-9d7a7640813e None admin] RBAC: Authorizing `identity:get_user(user_id=d30edb8c976e4642a7b873e1ad9ced36)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:25.580619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2f1f4-6ed4-4262-bc3b-9d7a7640813e None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:25.582352 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 549/1114] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:25 2022] GET /identity/v3/users/d30edb8c976e4642a7b873e1ad9ced36 => generated 286 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:25.588791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d1b80-4ecd-4441-bca4-9a2d6bf06191 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:25.694331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d1b80-4ecd-4441-bca4-9a2d6bf06191 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:25.721294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:25.729289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7623d9-54b4-4a0c-a593-33b8d2b87428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:25.729913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7623d9-54b4-4a0c-a593-33b8d2b87428 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:25.730220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7623d9-54b4-4a0c-a593-33b8d2b87428 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:25.730466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7623d9-54b4-4a0c-a593-33b8d2b87428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:25.803966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7623d9-54b4-4a0c-a593-33b8d2b87428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:25.805426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7623d9-54b4-4a0c-a593-33b8d2b87428 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:25.910975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 566/1115] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:08:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:08:25.917237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2f1f4-6ed4-4262-bc3b-9d7a7640813e None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:26.020528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:26.029862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2f1f4-6ed4-4262-bc3b-9d7a7640813e None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:26.066746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1ed59-d5e3-4f8b-9e35-08f9bd4626bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:26.067385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1ed59-d5e3-4f8b-9e35-08f9bd4626bd None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:26.067696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1ed59-d5e3-4f8b-9e35-08f9bd4626bd None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:26.067972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1ed59-d5e3-4f8b-9e35-08f9bd4626bd None admin] PATH_INFO: `/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:26.069106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1ed59-d5e3-4f8b-9e35-08f9bd4626bd None admin] RBAC: Authorizing `identity:get_project(project_id=8589ac3b9ead4fde8e2c2edc62907af5)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:26.070807 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1ed59-d5e3-4f8b-9e35-08f9bd4626bd None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:26.072547 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 550/1116] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:08:25 2022] GET /identity/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:26.080096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7623d9-54b4-4a0c-a593-33b8d2b87428 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:26.111569 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:26.196395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7623d9-54b4-4a0c-a593-33b8d2b87428 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:26.235057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ebba8-fd22-4e0e-9b4b-074235a97b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:26.235825 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ebba8-fd22-4e0e-9b4b-074235a97b2c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:26.236114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ebba8-fd22-4e0e-9b4b-074235a97b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:26.236387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ebba8-fd22-4e0e-9b4b-074235a97b2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:26.237122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ebba8-fd22-4e0e-9b4b-074235a97b2c None admin] RBAC: Adding query filter params (role.id=b9c95d84eaca4444b2c7b8b5eed61a59, scope.project.id=8589ac3b9ead4fde8e2c2edc62907af5, user.id=d30edb8c976e4642a7b873e1ad9ced36) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:26.237430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ebba8-fd22-4e0e-9b4b-074235a97b2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:26.238873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ebba8-fd22-4e0e-9b4b-074235a97b2c None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:26.245064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 567/1117] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:08:26 2022] GET /identity/v3/role_assignments?user.id=d30edb8c976e4642a7b873e1ad9ced36&scope.project.id=8589ac3b9ead4fde8e2c2edc62907af5&role.id=b9c95d84eaca4444b2c7b8b5eed61a59 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:28.446949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ab92a-ca25-43c3-ae2a-ee562505785b None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:28.447758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ab92a-ca25-43c3-ae2a-ee562505785b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:28.447758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ab92a-ca25-43c3-ae2a-ee562505785b None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:28.449169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 551/1118] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:28.458299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5281b5-e80b-4741-97ca-8e519848b270 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:28.459142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5281b5-e80b-4741-97ca-8e519848b270 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:28.459142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5281b5-e80b-4741-97ca-8e519848b270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:28.478942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-2d5281b5-e80b-4741-97ca-8e519848b270 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:28.480212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2d5281b5-e80b-4741-97ca-8e519848b270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:28.558977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 568/1119] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:08:28.833349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ab92a-ca25-43c3-ae2a-ee562505785b None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:28.958588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ab92a-ca25-43c3-ae2a-ee562505785b None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:28.999610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f3e7b-59d4-4122-a6eb-994fdd861578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:29.000273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f3e7b-59d4-4122-a6eb-994fdd861578 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:29.000553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f3e7b-59d4-4122-a6eb-994fdd861578 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:29.000832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f3e7b-59d4-4122-a6eb-994fdd861578 None admin] PATH_INFO: `/v3/roles/b9c95d84eaca4444b2c7b8b5eed61a59` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:29.002359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f3e7b-59d4-4122-a6eb-994fdd861578 None admin] RBAC: Authorizing `identity:get_role(role_id=b9c95d84eaca4444b2c7b8b5eed61a59)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:29.003802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f3e7b-59d4-4122-a6eb-994fdd861578 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:29.005269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 552/1120] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:28 2022] GET /identity/v3/roles/b9c95d84eaca4444b2c7b8b5eed61a59 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:29.011526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5281b5-e80b-4741-97ca-8e519848b270 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:29.042962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:29.133476 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5281b5-e80b-4741-97ca-8e519848b270 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:29.174911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7edfc-b5d3-441d-858e-38c30f40becf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:29.175595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7edfc-b5d3-441d-858e-38c30f40becf None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:29.175869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7edfc-b5d3-441d-858e-38c30f40becf None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:29.176151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7edfc-b5d3-441d-858e-38c30f40becf None admin] PATH_INFO: `/v3/users/d30edb8c976e4642a7b873e1ad9ced36` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:29.177346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7edfc-b5d3-441d-858e-38c30f40becf None admin] RBAC: Authorizing `identity:get_user(user_id=d30edb8c976e4642a7b873e1ad9ced36)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:29.178810 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7edfc-b5d3-441d-858e-38c30f40becf None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:29.180532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 569/1121] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:29 2022] GET /identity/v3/users/d30edb8c976e4642a7b873e1ad9ced36 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:29.186564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f3e7b-59d4-4122-a6eb-994fdd861578 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:29.216905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:29.292876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f3e7b-59d4-4122-a6eb-994fdd861578 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:29.332122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212f293-9cce-4916-82d8-0a75c9fa525b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:29.332961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212f293-9cce-4916-82d8-0a75c9fa525b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:29.333385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212f293-9cce-4916-82d8-0a75c9fa525b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:29.333635 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212f293-9cce-4916-82d8-0a75c9fa525b None admin] PATH_INFO: `/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:29.334990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212f293-9cce-4916-82d8-0a75c9fa525b None admin] RBAC: Authorizing `identity:get_project(project_id=8589ac3b9ead4fde8e2c2edc62907af5)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:29.336575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212f293-9cce-4916-82d8-0a75c9fa525b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:29.338395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 553/1122] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:08:29 2022] GET /identity/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:29.345319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7edfc-b5d3-441d-858e-38c30f40becf None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:29.450096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:29.468584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7edfc-b5d3-441d-858e-38c30f40becf None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:29.522587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2040ee-7c2e-42f3-b910-65a1b18aa621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:29.523787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2040ee-7c2e-42f3-b910-65a1b18aa621 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:29.524244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2040ee-7c2e-42f3-b910-65a1b18aa621 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:29.524740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2040ee-7c2e-42f3-b910-65a1b18aa621 None admin] PATH_INFO: `/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5/users/d30edb8c976e4642a7b873e1ad9ced36/roles/b9c95d84eaca4444b2c7b8b5eed61a59` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:29.528845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2040ee-7c2e-42f3-b910-65a1b18aa621 None admin] RBAC: Authorizing `identity:create_grant(project_id=8589ac3b9ead4fde8e2c2edc62907af5, user_id=d30edb8c976e4642a7b873e1ad9ced36, role_id=b9c95d84eaca4444b2c7b8b5eed61a59)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:29.532497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2040ee-7c2e-42f3-b910-65a1b18aa621 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:29.626549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 570/1123] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:08:29 2022] PUT /identity/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5/users/d30edb8c976e4642a7b873e1ad9ced36/roles/b9c95d84eaca4444b2c7b8b5eed61a59 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:08:31.918024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12755f-55da-4726-a466-98fe5494c07d None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:31.918619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12755f-55da-4726-a466-98fe5494c07d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:31.918619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12755f-55da-4726-a466-98fe5494c07d None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:31.921267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 554/1124] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:31.927719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426648a-dbc1-4e65-bc97-8ff19a422ac8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:31.928298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426648a-dbc1-4e65-bc97-8ff19a422ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:31.928298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426648a-dbc1-4e65-bc97-8ff19a422ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:31.949136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3426648a-dbc1-4e65-bc97-8ff19a422ac8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:31.950384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3426648a-dbc1-4e65-bc97-8ff19a422ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:32.036919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 571/1125] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:32.310454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12755f-55da-4726-a466-98fe5494c07d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:32.427677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:32.434987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12755f-55da-4726-a466-98fe5494c07d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:32.475871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea26eeb-ffb8-47a9-b4ac-b63ffc8c73bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:32.476525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea26eeb-ffb8-47a9-b4ac-b63ffc8c73bf None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:32.476816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea26eeb-ffb8-47a9-b4ac-b63ffc8c73bf None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:32.477096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea26eeb-ffb8-47a9-b4ac-b63ffc8c73bf None admin] PATH_INFO: `/v3/roles/b9c95d84eaca4444b2c7b8b5eed61a59` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:32.478569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea26eeb-ffb8-47a9-b4ac-b63ffc8c73bf None admin] RBAC: Authorizing `identity:get_role(role_id=b9c95d84eaca4444b2c7b8b5eed61a59)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:32.479999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea26eeb-ffb8-47a9-b4ac-b63ffc8c73bf None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:32.481403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 555/1126] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:32 2022] GET /identity/v3/roles/b9c95d84eaca4444b2c7b8b5eed61a59 => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:32.488074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426648a-dbc1-4e65-bc97-8ff19a422ac8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:32.590004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:32.610931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426648a-dbc1-4e65-bc97-8ff19a422ac8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:32.648039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9dba0-23ef-43bb-83bf-d822127ea19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:32.648680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9dba0-23ef-43bb-83bf-d822127ea19d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:32.648965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9dba0-23ef-43bb-83bf-d822127ea19d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:32.649237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9dba0-23ef-43bb-83bf-d822127ea19d None admin] PATH_INFO: `/v3/users/d30edb8c976e4642a7b873e1ad9ced36` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:32.650408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9dba0-23ef-43bb-83bf-d822127ea19d None admin] RBAC: Authorizing `identity:get_user(user_id=d30edb8c976e4642a7b873e1ad9ced36)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:32.651992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9dba0-23ef-43bb-83bf-d822127ea19d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:32.653721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 572/1127] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:32 2022] GET /identity/v3/users/d30edb8c976e4642a7b873e1ad9ced36 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:32.660508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea26eeb-ffb8-47a9-b4ac-b63ffc8c73bf None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:32.771805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea26eeb-ffb8-47a9-b4ac-b63ffc8c73bf None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:32.810377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901c2d6-6930-46be-885c-05d37b1b7060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:32.810999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901c2d6-6930-46be-885c-05d37b1b7060 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:32.811279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901c2d6-6930-46be-885c-05d37b1b7060 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:32.811616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901c2d6-6930-46be-885c-05d37b1b7060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:32.850777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:32.891643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901c2d6-6930-46be-885c-05d37b1b7060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:32.893027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901c2d6-6930-46be-885c-05d37b1b7060 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:32.991777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 556/1128] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:08:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 23 01:08:33.001262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9dba0-23ef-43bb-83bf-d822127ea19d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:33.112331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9dba0-23ef-43bb-83bf-d822127ea19d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:33.146301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc95a0b-dae3-4c6e-92b8-fa7d15cfe3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:33.146935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95a0b-dae3-4c6e-92b8-fa7d15cfe3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:33.147282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95a0b-dae3-4c6e-92b8-fa7d15cfe3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:33.147578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc95a0b-dae3-4c6e-92b8-fa7d15cfe3c7 None admin] PATH_INFO: `/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:33.148712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95a0b-dae3-4c6e-92b8-fa7d15cfe3c7 None admin] RBAC: Authorizing `identity:get_project(project_id=8589ac3b9ead4fde8e2c2edc62907af5)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:33.150147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc95a0b-dae3-4c6e-92b8-fa7d15cfe3c7 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:33.151869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 573/1129] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:08:32 2022] GET /identity/v3/projects/8589ac3b9ead4fde8e2c2edc62907af5 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:33.159094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901c2d6-6930-46be-885c-05d37b1b7060 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:33.246594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:33.297491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901c2d6-6930-46be-885c-05d37b1b7060 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:33.340113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561032bd-6e9e-4dc0-abaf-9adf5078d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:33.340761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561032bd-6e9e-4dc0-abaf-9adf5078d3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:33.341049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561032bd-6e9e-4dc0-abaf-9adf5078d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:33.341326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561032bd-6e9e-4dc0-abaf-9adf5078d3e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:33.342095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561032bd-6e9e-4dc0-abaf-9adf5078d3e3 None admin] RBAC: Adding query filter params (role.id=b9c95d84eaca4444b2c7b8b5eed61a59, scope.project.id=8589ac3b9ead4fde8e2c2edc62907af5, user.id=d30edb8c976e4642a7b873e1ad9ced36) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:33.342410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561032bd-6e9e-4dc0-abaf-9adf5078d3e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:33.343999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561032bd-6e9e-4dc0-abaf-9adf5078d3e3 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:33.350895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 557/1130] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:08:33 2022] GET /identity/v3/role_assignments?user.id=d30edb8c976e4642a7b873e1ad9ced36&scope.project.id=8589ac3b9ead4fde8e2c2edc62907af5&role.id=b9c95d84eaca4444b2c7b8b5eed61a59 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:35.608098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888beec-aafa-463d-a862-37c11fc5d256 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:35.608687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888beec-aafa-463d-a862-37c11fc5d256 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:35.608687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d888beec-aafa-463d-a862-37c11fc5d256 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:35.610087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 574/1131] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:35.616395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f6ca3-f5dd-4970-b331-48da0ccdec72 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:35.617203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f6ca3-f5dd-4970-b331-48da0ccdec72 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:35.617203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f6ca3-f5dd-4970-b331-48da0ccdec72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:35.637262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-c06f6ca3-f5dd-4970-b331-48da0ccdec72 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:35.638545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c06f6ca3-f5dd-4970-b331-48da0ccdec72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:35.738790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 558/1132] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:36.013115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888beec-aafa-463d-a862-37c11fc5d256 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:36.034863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:36.139866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d888beec-aafa-463d-a862-37c11fc5d256 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:36.182777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b9eb2-8edc-4f81-bec2-9646ca7b260e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:36.183468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b9eb2-8edc-4f81-bec2-9646ca7b260e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:36.183771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b9eb2-8edc-4f81-bec2-9646ca7b260e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:36.184043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b9eb2-8edc-4f81-bec2-9646ca7b260e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:36.185253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b9eb2-8edc-4f81-bec2-9646ca7b260e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:36.186710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b9eb2-8edc-4f81-bec2-9646ca7b260e None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:36.189036 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 575/1133] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:36.195419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f6ca3-f5dd-4970-b331-48da0ccdec72 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:36.272747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:36.322566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f6ca3-f5dd-4970-b331-48da0ccdec72 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:36.365841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af88ac9-7256-48e4-b2d3-ebb723457084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:36.366465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af88ac9-7256-48e4-b2d3-ebb723457084 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:36.366742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af88ac9-7256-48e4-b2d3-ebb723457084 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:36.367020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af88ac9-7256-48e4-b2d3-ebb723457084 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:36.367726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af88ac9-7256-48e4-b2d3-ebb723457084 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:36.369195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af88ac9-7256-48e4-b2d3-ebb723457084 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:36.467619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 559/1134] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:08:36 2022] POST /identity/v3/projects => generated 311 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:08:38.730136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de266b-b95b-4f95-8d88-a47d30eed38a None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:38.730699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de266b-b95b-4f95-8d88-a47d30eed38a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:38.730699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de266b-b95b-4f95-8d88-a47d30eed38a None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:38.732111 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 576/1135] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:38.738144 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff9c28-c6fb-4743-9c6f-99207580f828 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:38.738591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff9c28-c6fb-4743-9c6f-99207580f828 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:38.738684 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff9c28-c6fb-4743-9c6f-99207580f828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:38.756918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-97ff9c28-c6fb-4743-9c6f-99207580f828 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:38.758128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-97ff9c28-c6fb-4743-9c6f-99207580f828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:38.824629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 560/1136] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:39.103722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de266b-b95b-4f95-8d88-a47d30eed38a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:39.188150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:39.205861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de266b-b95b-4f95-8d88-a47d30eed38a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:39.241270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45ab58-9185-4cfe-99b8-498c51c09f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:39.241905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45ab58-9185-4cfe-99b8-498c51c09f44 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:39.242186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45ab58-9185-4cfe-99b8-498c51c09f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:39.242455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45ab58-9185-4cfe-99b8-498c51c09f44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:39.243598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45ab58-9185-4cfe-99b8-498c51c09f44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:39.245037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45ab58-9185-4cfe-99b8-498c51c09f44 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:39.246660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 577/1137] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:08:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:39.253253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff9c28-c6fb-4743-9c6f-99207580f828 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:39.299649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:39.365711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff9c28-c6fb-4743-9c6f-99207580f828 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:39.403772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f874c21-01b8-43ce-8ae5-25fa937f8177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:39.404382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f874c21-01b8-43ce-8ae5-25fa937f8177 None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:39.404663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f874c21-01b8-43ce-8ae5-25fa937f8177 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:39.404931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f874c21-01b8-43ce-8ae5-25fa937f8177 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:39.405624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f874c21-01b8-43ce-8ae5-25fa937f8177 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:39.407065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f874c21-01b8-43ce-8ae5-25fa937f8177 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:39.539848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 561/1138] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:08:39 2022] POST /identity/v3/users => generated 286 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:08:41.771042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d71169-0b6d-4e1f-a9a2-d715f83c7527 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:41.771760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d71169-0b6d-4e1f-a9a2-d715f83c7527 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:41.771760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d71169-0b6d-4e1f-a9a2-d715f83c7527 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:41.773250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 578/1139] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:41.779044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3ad2-35e5-4147-a522-896d32c0f93e None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:41.779542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3ad2-35e5-4147-a522-896d32c0f93e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:41.779683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3ad2-35e5-4147-a522-896d32c0f93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:41.797741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-bfbd3ad2-35e5-4147-a522-896d32c0f93e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:41.798946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-bfbd3ad2-35e5-4147-a522-896d32c0f93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:41.865722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 562/1140] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:08:42.139808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d71169-0b6d-4e1f-a9a2-d715f83c7527 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:42.244682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d71169-0b6d-4e1f-a9a2-d715f83c7527 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:42.279641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088ffa0-4528-4149-9ba5-dc298c2cc884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:42.280272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088ffa0-4528-4149-9ba5-dc298c2cc884 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:42.282418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088ffa0-4528-4149-9ba5-dc298c2cc884 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:42.282418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088ffa0-4528-4149-9ba5-dc298c2cc884 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:42.291661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3088ffa0-4528-4149-9ba5-dc298c2cc884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-3088ffa0-4528-4149-9ba5-dc298c2cc884 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:08:42.297299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:42.298546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:42.299845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 579/1141] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:08:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:42.304913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd3ad2-35e5-4147-a522-896d32c0f93e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:42.332628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:42.405194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd3ad2-35e5-4147-a522-896d32c0f93e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:42.439055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678c0a2-6619-43ba-9b19-9193de144def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:42.439695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678c0a2-6619-43ba-9b19-9193de144def None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:42.439980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678c0a2-6619-43ba-9b19-9193de144def None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:42.440254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678c0a2-6619-43ba-9b19-9193de144def None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:42.440994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678c0a2-6619-43ba-9b19-9193de144def None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:42.441290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678c0a2-6619-43ba-9b19-9193de144def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:42.442665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678c0a2-6619-43ba-9b19-9193de144def None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:42.451569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 563/1142] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:08:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:42.458213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088ffa0-4528-4149-9ba5-dc298c2cc884 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:42.473440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:42.555420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088ffa0-4528-4149-9ba5-dc298c2cc884 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:42.590231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94006a9-9ade-4f69-bb33-2602b6518b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:42.590862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94006a9-9ade-4f69-bb33-2602b6518b48 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:42.591139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94006a9-9ade-4f69-bb33-2602b6518b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:42.591407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94006a9-9ade-4f69-bb33-2602b6518b48 None admin] PATH_INFO: `/v3/users/11e85b5ed61541cba02c5e2c70d08fcc` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:42.604721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94006a9-9ade-4f69-bb33-2602b6518b48 None admin] RBAC: Authorizing `identity:get_user(user_id=11e85b5ed61541cba02c5e2c70d08fcc)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:42.606199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94006a9-9ade-4f69-bb33-2602b6518b48 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:42.607984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 580/1143] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:42 2022] GET /identity/v3/users/11e85b5ed61541cba02c5e2c70d08fcc => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:42.614353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678c0a2-6619-43ba-9b19-9193de144def None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:42.703113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:42.715149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678c0a2-6619-43ba-9b19-9193de144def None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:42.750632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8470382-a930-41cf-b1fb-a9a336292c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:42.751253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8470382-a930-41cf-b1fb-a9a336292c75 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:42.751538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8470382-a930-41cf-b1fb-a9a336292c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:42.751851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8470382-a930-41cf-b1fb-a9a336292c75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:42.822399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8470382-a930-41cf-b1fb-a9a336292c75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:42.823930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8470382-a930-41cf-b1fb-a9a336292c75 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:42.924044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:42.936462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 564/1144] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:08:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:08:42.943234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94006a9-9ade-4f69-bb33-2602b6518b48 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:43.015497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:43.053324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94006a9-9ade-4f69-bb33-2602b6518b48 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:43.089312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5cc1c-4c19-48cd-a7de-8262356d0665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:43.090010 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5cc1c-4c19-48cd-a7de-8262356d0665 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:43.090293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5cc1c-4c19-48cd-a7de-8262356d0665 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:43.090562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5cc1c-4c19-48cd-a7de-8262356d0665 None admin] PATH_INFO: `/v3/projects/4eae43a4fecc4030850ac10b63fab280` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:43.091718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5cc1c-4c19-48cd-a7de-8262356d0665 None admin] RBAC: Authorizing `identity:get_project(project_id=4eae43a4fecc4030850ac10b63fab280)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:43.093663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5cc1c-4c19-48cd-a7de-8262356d0665 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:43.095342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 581/1145] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:08:42 2022] GET /identity/v3/projects/4eae43a4fecc4030850ac10b63fab280 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:43.101578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8470382-a930-41cf-b1fb-a9a336292c75 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:43.201184 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8470382-a930-41cf-b1fb-a9a336292c75 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:43.235575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9e0e3-6619-44bf-8893-4a9579a23877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:43.236224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9e0e3-6619-44bf-8893-4a9579a23877 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:43.236506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9e0e3-6619-44bf-8893-4a9579a23877 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:43.236781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9e0e3-6619-44bf-8893-4a9579a23877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:43.237510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9e0e3-6619-44bf-8893-4a9579a23877 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=4eae43a4fecc4030850ac10b63fab280, user.id=11e85b5ed61541cba02c5e2c70d08fcc) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:43.237876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9e0e3-6619-44bf-8893-4a9579a23877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:43.239241 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9e0e3-6619-44bf-8893-4a9579a23877 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:43.244716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 565/1146] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:08:43 2022] GET /identity/v3/role_assignments?user.id=11e85b5ed61541cba02c5e2c70d08fcc&scope.project.id=4eae43a4fecc4030850ac10b63fab280&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:45.407256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4af15-d92f-4bcd-b26a-4d84d0b9c235 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:45.407821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4af15-d92f-4bcd-b26a-4d84d0b9c235 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:45.407954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4af15-d92f-4bcd-b26a-4d84d0b9c235 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:45.409232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 582/1147] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:45.414615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae042b7-09be-42e6-aae2-d0c52d001684 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:45.415019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae042b7-09be-42e6-aae2-d0c52d001684 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:45.415164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae042b7-09be-42e6-aae2-d0c52d001684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:45.433751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-4ae042b7-09be-42e6-aae2-d0c52d001684 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:45.434957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4ae042b7-09be-42e6-aae2-d0c52d001684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:45.484136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:45.504256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 566/1148] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:45.776918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4af15-d92f-4bcd-b26a-4d84d0b9c235 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:45.886533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4af15-d92f-4bcd-b26a-4d84d0b9c235 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:45.924369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cd298-8dc4-4ad1-8340-2172c6b72ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:45.925014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cd298-8dc4-4ad1-8340-2172c6b72ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:45.925295 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cd298-8dc4-4ad1-8340-2172c6b72ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:45.925569 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cd298-8dc4-4ad1-8340-2172c6b72ac0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:45.933425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:45.937291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cd298-8dc4-4ad1-8340-2172c6b72ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-3e7cd298-8dc4-4ad1-8340-2172c6b72ac0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:08:45.942159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:45.943139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:45.944238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 583/1149] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:08:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:45.949870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae042b7-09be-42e6-aae2-d0c52d001684 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:46.068546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae042b7-09be-42e6-aae2-d0c52d001684 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:46.106073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4ae5b-9de5-4715-8fd3-bdd482884c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:46.106693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4ae5b-9de5-4715-8fd3-bdd482884c52 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:46.106972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4ae5b-9de5-4715-8fd3-bdd482884c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:46.107240 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4ae5b-9de5-4715-8fd3-bdd482884c52 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:46.107945 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4ae5b-9de5-4715-8fd3-bdd482884c52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:46.108248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4ae5b-9de5-4715-8fd3-bdd482884c52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:46.109644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4ae5b-9de5-4715-8fd3-bdd482884c52 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:46.118905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 567/1150] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:08:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:46.125671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cd298-8dc4-4ad1-8340-2172c6b72ac0 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:46.231190 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cd298-8dc4-4ad1-8340-2172c6b72ac0 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:46.274171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b8968-f03e-44ae-b5ec-52824bcb8d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:46.274812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b8968-f03e-44ae-b5ec-52824bcb8d39 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:46.275094 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b8968-f03e-44ae-b5ec-52824bcb8d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:46.275367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b8968-f03e-44ae-b5ec-52824bcb8d39 None admin] PATH_INFO: `/v3/users/11e85b5ed61541cba02c5e2c70d08fcc` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:46.276735 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b8968-f03e-44ae-b5ec-52824bcb8d39 None admin] RBAC: Authorizing `identity:get_user(user_id=11e85b5ed61541cba02c5e2c70d08fcc)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:46.278188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b8968-f03e-44ae-b5ec-52824bcb8d39 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:46.279871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 584/1151] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:46 2022] GET /identity/v3/users/11e85b5ed61541cba02c5e2c70d08fcc => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:46.286824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4ae5b-9de5-4715-8fd3-bdd482884c52 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:46.341065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:46.404888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4ae5b-9de5-4715-8fd3-bdd482884c52 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:46.442031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a6d91-ac8a-4653-869c-7e0ff7106169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:46.442693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a6d91-ac8a-4653-869c-7e0ff7106169 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:46.442950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a6d91-ac8a-4653-869c-7e0ff7106169 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:46.443237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a6d91-ac8a-4653-869c-7e0ff7106169 None admin] PATH_INFO: `/v3/projects/4eae43a4fecc4030850ac10b63fab280` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:46.444429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a6d91-ac8a-4653-869c-7e0ff7106169 None admin] RBAC: Authorizing `identity:get_project(project_id=4eae43a4fecc4030850ac10b63fab280)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:46.445886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a6d91-ac8a-4653-869c-7e0ff7106169 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:46.447891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 568/1152] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:08:46 2022] GET /identity/v3/projects/4eae43a4fecc4030850ac10b63fab280 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:46.454216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b8968-f03e-44ae-b5ec-52824bcb8d39 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:46.522801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:46.555955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b8968-f03e-44ae-b5ec-52824bcb8d39 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:46.593475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94b81c-a70b-4623-8f1a-31953d96d4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:46.594074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94b81c-a70b-4623-8f1a-31953d96d4d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:46.594353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94b81c-a70b-4623-8f1a-31953d96d4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:46.594628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94b81c-a70b-4623-8f1a-31953d96d4d4 None admin] PATH_INFO: `/v3/projects/4eae43a4fecc4030850ac10b63fab280/users/11e85b5ed61541cba02c5e2c70d08fcc/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:46.596697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94b81c-a70b-4623-8f1a-31953d96d4d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4eae43a4fecc4030850ac10b63fab280, user_id=11e85b5ed61541cba02c5e2c70d08fcc, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:46.598163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc94b81c-a70b-4623-8f1a-31953d96d4d4 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:46.785584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 585/1153] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:08:46 2022] PUT /identity/v3/projects/4eae43a4fecc4030850ac10b63fab280/users/11e85b5ed61541cba02c5e2c70d08fcc/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:08:48.955723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3bb89-d05d-4efb-9260-61cfe41266c3 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:48.956252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3bb89-d05d-4efb-9260-61cfe41266c3 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:48.956252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd3bb89-d05d-4efb-9260-61cfe41266c3 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:48.957662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 569/1154] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:48.965523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e30e57-b6a6-4bb3-bbdf-5cb4c14c5199 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:48.965523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e30e57-b6a6-4bb3-bbdf-5cb4c14c5199 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:48.965523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e30e57-b6a6-4bb3-bbdf-5cb4c14c5199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:48.982645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b6e30e57-b6a6-4bb3-bbdf-5cb4c14c5199 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:48.983889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b6e30e57-b6a6-4bb3-bbdf-5cb4c14c5199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:49.050966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 586/1155] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:49.325070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3bb89-d05d-4efb-9260-61cfe41266c3 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:49.433005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd3bb89-d05d-4efb-9260-61cfe41266c3 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:49.446006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:49.469210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdeda9-e68b-4872-b7c1-5dfdd9cec9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:49.469841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdeda9-e68b-4872-b7c1-5dfdd9cec9fd None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:49.470116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdeda9-e68b-4872-b7c1-5dfdd9cec9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:49.470386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcdeda9-e68b-4872-b7c1-5dfdd9cec9fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:49.480904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcdeda9-e68b-4872-b7c1-5dfdd9cec9fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-fbcdeda9-e68b-4872-b7c1-5dfdd9cec9fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:08:49.485744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:49.486819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:08:49.487721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:08:49.488643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 570/1156] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:08:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:49.493091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e30e57-b6a6-4bb3-bbdf-5cb4c14c5199 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:49.551842 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:49.598333 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e30e57-b6a6-4bb3-bbdf-5cb4c14c5199 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:49.635491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff40ea4-dc64-472d-83c9-b213bc8c1581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:49.636187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff40ea4-dc64-472d-83c9-b213bc8c1581 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:49.636467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff40ea4-dc64-472d-83c9-b213bc8c1581 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:49.636743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff40ea4-dc64-472d-83c9-b213bc8c1581 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:49.637430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff40ea4-dc64-472d-83c9-b213bc8c1581 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:49.637730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff40ea4-dc64-472d-83c9-b213bc8c1581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:49.639107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff40ea4-dc64-472d-83c9-b213bc8c1581 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:49.649251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 587/1157] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:08:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:49.655424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdeda9-e68b-4872-b7c1-5dfdd9cec9fd None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:49.758589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcdeda9-e68b-4872-b7c1-5dfdd9cec9fd None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:49.795284 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4fa68-f3d8-4c06-bfed-343c5c209a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:49.795973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4fa68-f3d8-4c06-bfed-343c5c209a2d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:49.796284 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4fa68-f3d8-4c06-bfed-343c5c209a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:49.796559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4fa68-f3d8-4c06-bfed-343c5c209a2d None admin] PATH_INFO: `/v3/users/11e85b5ed61541cba02c5e2c70d08fcc` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:49.797793 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4fa68-f3d8-4c06-bfed-343c5c209a2d None admin] RBAC: Authorizing `identity:get_user(user_id=11e85b5ed61541cba02c5e2c70d08fcc)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:49.799274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4fa68-f3d8-4c06-bfed-343c5c209a2d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:49.801029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 571/1158] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:08:49 2022] GET /identity/v3/users/11e85b5ed61541cba02c5e2c70d08fcc => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:49.807192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff40ea4-dc64-472d-83c9-b213bc8c1581 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:49.910042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff40ea4-dc64-472d-83c9-b213bc8c1581 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:49.927832 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:49.946361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557b15b-8f85-4f4a-b749-9cd59d2cbd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:49.946993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557b15b-8f85-4f4a-b749-9cd59d2cbd75 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:49.947270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557b15b-8f85-4f4a-b749-9cd59d2cbd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:49.947542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557b15b-8f85-4f4a-b749-9cd59d2cbd75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:50.028372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557b15b-8f85-4f4a-b749-9cd59d2cbd75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:50.029880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557b15b-8f85-4f4a-b749-9cd59d2cbd75 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:50.133201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 588/1159] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:08:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:08:50.139522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4fa68-f3d8-4c06-bfed-343c5c209a2d None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:50.181993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:50.240465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4fa68-f3d8-4c06-bfed-343c5c209a2d None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:50.285715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22307f0-8d43-43fb-ba1a-2ed4b03c100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:50.286351 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22307f0-8d43-43fb-ba1a-2ed4b03c100b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:50.286630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22307f0-8d43-43fb-ba1a-2ed4b03c100b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:50.286904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22307f0-8d43-43fb-ba1a-2ed4b03c100b None admin] PATH_INFO: `/v3/projects/4eae43a4fecc4030850ac10b63fab280` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:50.288079 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22307f0-8d43-43fb-ba1a-2ed4b03c100b None admin] RBAC: Authorizing `identity:get_project(project_id=4eae43a4fecc4030850ac10b63fab280)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:50.289541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22307f0-8d43-43fb-ba1a-2ed4b03c100b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:50.291405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 572/1160] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:08:50 2022] GET /identity/v3/projects/4eae43a4fecc4030850ac10b63fab280 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:08:50.298438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557b15b-8f85-4f4a-b749-9cd59d2cbd75 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:50.318291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:50.410254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557b15b-8f85-4f4a-b749-9cd59d2cbd75 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:50.453196 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29d2e9-55b2-435e-aa64-b2cfb4f633c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:50.453741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29d2e9-55b2-435e-aa64-b2cfb4f633c9 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:50.454025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29d2e9-55b2-435e-aa64-b2cfb4f633c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:50.454299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29d2e9-55b2-435e-aa64-b2cfb4f633c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:50.455044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29d2e9-55b2-435e-aa64-b2cfb4f633c9 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=4eae43a4fecc4030850ac10b63fab280, user.id=11e85b5ed61541cba02c5e2c70d08fcc) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:50.455355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29d2e9-55b2-435e-aa64-b2cfb4f633c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:50.456838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29d2e9-55b2-435e-aa64-b2cfb4f633c9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:50.463244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 589/1161] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:08:50 2022] GET /identity/v3/role_assignments?user.id=11e85b5ed61541cba02c5e2c70d08fcc&scope.project.id=4eae43a4fecc4030850ac10b63fab280&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:52.679420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636d9c4-46ae-482e-97a2-a9abbb68c7fd None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:52.679998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636d9c4-46ae-482e-97a2-a9abbb68c7fd None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:52.679998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636d9c4-46ae-482e-97a2-a9abbb68c7fd None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:52.681368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 573/1162] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:52.686996 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f264-e7b2-4f33-a34c-b717f3fc6632 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:52.687513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f264-e7b2-4f33-a34c-b717f3fc6632 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:52.687640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f264-e7b2-4f33-a34c-b717f3fc6632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:52.706785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-af01f264-e7b2-4f33-a34c-b717f3fc6632 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:52.708065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-af01f264-e7b2-4f33-a34c-b717f3fc6632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:52.783383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 590/1163] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:53.052826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636d9c4-46ae-482e-97a2-a9abbb68c7fd None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:53.160369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636d9c4-46ae-482e-97a2-a9abbb68c7fd None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:53.170672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:53.195378 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e19208-34f6-4aa3-8d96-eccc2748e006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:53.196015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e19208-34f6-4aa3-8d96-eccc2748e006 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:53.196551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e19208-34f6-4aa3-8d96-eccc2748e006 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:53.196825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e19208-34f6-4aa3-8d96-eccc2748e006 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:53.267126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e19208-34f6-4aa3-8d96-eccc2748e006 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:53.268619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e19208-34f6-4aa3-8d96-eccc2748e006 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:53.368702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 574/1164] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:08:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:08:53.375325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01f264-e7b2-4f33-a34c-b717f3fc6632 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:53.389676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:53.475585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01f264-e7b2-4f33-a34c-b717f3fc6632 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:53.512107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8fe11-b20f-4539-a873-6863730b2709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:53.512768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8fe11-b20f-4539-a873-6863730b2709 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:53.513043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8fe11-b20f-4539-a873-6863730b2709 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:53.513315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8fe11-b20f-4539-a873-6863730b2709 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:53.521443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8fe11-b20f-4539-a873-6863730b2709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:53.522928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b8fe11-b20f-4539-a873-6863730b2709 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-35b8fe11-b20f-4539-a873-6863730b2709 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:08:53.536050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:08:53.537236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:08:53.538204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 591/1165] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Sep 23 01:08:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:08:53.543619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e19208-34f6-4aa3-8d96-eccc2748e006 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:53.555874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:53.656859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e19208-34f6-4aa3-8d96-eccc2748e006 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:53.694644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e6d11-a4e1-4d17-a943-fc10086754a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:53.695667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e6d11-a4e1-4d17-a943-fc10086754a7 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:53.695667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e6d11-a4e1-4d17-a943-fc10086754a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:53.695867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e6d11-a4e1-4d17-a943-fc10086754a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:53.696695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e6d11-a4e1-4d17-a943-fc10086754a7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:08:53.696999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e6d11-a4e1-4d17-a943-fc10086754a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:53.698370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e6d11-a4e1-4d17-a943-fc10086754a7 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:53.706758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 575/1166] 10.222.0.64 () {56 vars in 1115 bytes} [Fri Sep 23 01:08:53 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:55.909667 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a4485-b5ee-4529-a209-e840e369e3e6 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:55.910262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a4485-b5ee-4529-a209-e840e369e3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:55.910262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a4485-b5ee-4529-a209-e840e369e3e6 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:55.911601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 592/1167] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:55.917102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12d3c6-7181-4b83-95a2-bb83da659254 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:55.917561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12d3c6-7181-4b83-95a2-bb83da659254 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:55.917667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12d3c6-7181-4b83-95a2-bb83da659254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:55.936418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-dd12d3c6-7181-4b83-95a2-bb83da659254 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:55.937717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-dd12d3c6-7181-4b83-95a2-bb83da659254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:56.009032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 576/1168] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:56.277848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a4485-b5ee-4529-a209-e840e369e3e6 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:56.355150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:56.386495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a4485-b5ee-4529-a209-e840e369e3e6 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:56.422473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1caa20-f1fc-4561-b5f4-cb0026ddf1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:56.423079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1caa20-f1fc-4561-b5f4-cb0026ddf1c8 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:56.423358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1caa20-f1fc-4561-b5f4-cb0026ddf1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:56.423660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1caa20-f1fc-4561-b5f4-cb0026ddf1c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:56.424339 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1caa20-f1fc-4561-b5f4-cb0026ddf1c8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:56.425713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1caa20-f1fc-4561-b5f4-cb0026ddf1c8 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:56.548012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 593/1169] 10.222.0.64 () {60 vars in 1138 bytes} [Fri Sep 23 01:08:56 2022] POST /identity/v3/domains => generated 267 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:08:58.755318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134427e-d93a-49d8-8042-942532661dfe None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:58.756028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134427e-d93a-49d8-8042-942532661dfe None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:58.756028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134427e-d93a-49d8-8042-942532661dfe None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:58.757426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 577/1170] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:08:58.762913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6d823-34b8-40b8-8f03-68ec9826105d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:58.763418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6d823-34b8-40b8-8f03-68ec9826105d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:58.763519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6d823-34b8-40b8-8f03-68ec9826105d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:58.782364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b0a6d823-34b8-40b8-8f03-68ec9826105d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:08:58.783605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b0a6d823-34b8-40b8-8f03-68ec9826105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:08:58.852352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 594/1171] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:08:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:08:59.121371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134427e-d93a-49d8-8042-942532661dfe None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:59.124369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:59.230105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134427e-d93a-49d8-8042-942532661dfe None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:59.264765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de29f7-49eb-4fd7-9667-e6ea1f6904bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:59.265938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de29f7-49eb-4fd7-9667-e6ea1f6904bc None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:59.266219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de29f7-49eb-4fd7-9667-e6ea1f6904bc None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:59.266496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de29f7-49eb-4fd7-9667-e6ea1f6904bc None admin] PATH_INFO: `/v3/domains/5b415f7911c048999fa29aeb18914ab9` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:59.279863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de29f7-49eb-4fd7-9667-e6ea1f6904bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=5b415f7911c048999fa29aeb18914ab9)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:59.283180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de29f7-49eb-4fd7-9667-e6ea1f6904bc None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:59.286285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 578/1172] 10.222.0.64 () {56 vars in 1183 bytes} [Fri Sep 23 01:08:59 2022] GET /identity/v3/domains/5b415f7911c048999fa29aeb18914ab9 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:08:59.292348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6d823-34b8-40b8-8f03-68ec9826105d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:08:59.355489 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:08:59.394274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6d823-34b8-40b8-8f03-68ec9826105d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:08:59.429200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca9cf84-812a-4aae-afff-84abc43b7a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:08:59.429818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9cf84-812a-4aae-afff-84abc43b7a95 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:08:59.430101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9cf84-812a-4aae-afff-84abc43b7a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:08:59.430375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca9cf84-812a-4aae-afff-84abc43b7a95 None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:08:59.431044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9cf84-812a-4aae-afff-84abc43b7a95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:08:59.432515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca9cf84-812a-4aae-afff-84abc43b7a95 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:08:59.518936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 595/1173] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:08:59 2022] POST /identity/v3/projects => generated 361 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:09:01.736647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5528b-7c13-4ca2-bac4-acc95def367e None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:01.737206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5528b-7c13-4ca2-bac4-acc95def367e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:01.737206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5528b-7c13-4ca2-bac4-acc95def367e None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:01.738544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 579/1174] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:09:01.743815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf3aa6-df7b-4e6e-8cce-488d0493049f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:01.744248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf3aa6-df7b-4e6e-8cce-488d0493049f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:01.744370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf3aa6-df7b-4e6e-8cce-488d0493049f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:01.762810 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c8bf3aa6-df7b-4e6e-8cce-488d0493049f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:09:01.764045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c8bf3aa6-df7b-4e6e-8cce-488d0493049f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:09:01.830647 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 596/1175] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:09:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:09:02.104599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5528b-7c13-4ca2-bac4-acc95def367e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:02.178504 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:02.220758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5528b-7c13-4ca2-bac4-acc95def367e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:02.255819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bad819-1372-4901-a5ba-0859f4d47907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:02.256441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bad819-1372-4901-a5ba-0859f4d47907 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:02.256714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bad819-1372-4901-a5ba-0859f4d47907 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:02.256985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bad819-1372-4901-a5ba-0859f4d47907 None admin] PATH_INFO: `/v3/domains/5b415f7911c048999fa29aeb18914ab9` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:02.258104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bad819-1372-4901-a5ba-0859f4d47907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5b415f7911c048999fa29aeb18914ab9)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:02.259524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bad819-1372-4901-a5ba-0859f4d47907 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:02.261166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 580/1176] 10.222.0.64 () {56 vars in 1183 bytes} [Fri Sep 23 01:09:02 2022] GET /identity/v3/domains/5b415f7911c048999fa29aeb18914ab9 => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:02.267184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf3aa6-df7b-4e6e-8cce-488d0493049f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:02.317761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:02.372015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf3aa6-df7b-4e6e-8cce-488d0493049f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:02.408062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c57703-fa82-4441-bf2d-9705b104bed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:02.408670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c57703-fa82-4441-bf2d-9705b104bed9 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:02.408945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c57703-fa82-4441-bf2d-9705b104bed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:02.409216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c57703-fa82-4441-bf2d-9705b104bed9 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:02.409878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c57703-fa82-4441-bf2d-9705b104bed9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:02.411302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c57703-fa82-4441-bf2d-9705b104bed9 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:02.534793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 597/1177] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:09:02 2022] POST /identity/v3/users => generated 311 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:09:04.765106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da15d56-21b6-4dc2-8202-2d46a2135360 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:04.765695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da15d56-21b6-4dc2-8202-2d46a2135360 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:04.765695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da15d56-21b6-4dc2-8202-2d46a2135360 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:04.767096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 581/1178] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:09:04.772543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa42db-852b-45f8-a221-0aa4ff3709a1 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:04.772982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa42db-852b-45f8-a221-0aa4ff3709a1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:04.773085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa42db-852b-45f8-a221-0aa4ff3709a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:04.791442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-2eaa42db-852b-45f8-a221-0aa4ff3709a1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:09:04.792662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2eaa42db-852b-45f8-a221-0aa4ff3709a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:09:04.860978 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 598/1179] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:09:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:09:05.140469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da15d56-21b6-4dc2-8202-2d46a2135360 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:05.246533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da15d56-21b6-4dc2-8202-2d46a2135360 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:05.275828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:05.282395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cdd88-eb8a-41ee-acb0-2e56d440ce46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:05.283061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cdd88-eb8a-41ee-acb0-2e56d440ce46 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:05.283346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cdd88-eb8a-41ee-acb0-2e56d440ce46 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:05.283643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cdd88-eb8a-41ee-acb0-2e56d440ce46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:05.293812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cdd88-eb8a-41ee-acb0-2e56d440ce46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-1d9cdd88-eb8a-41ee-acb0-2e56d440ce46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:09:05.298700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:09:05.299927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:09:05.301263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 582/1180] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:09:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:09:05.306021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa42db-852b-45f8-a221-0aa4ff3709a1 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:05.336727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:05.404816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa42db-852b-45f8-a221-0aa4ff3709a1 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:05.438282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ba75f-606d-4c2d-9778-27d92abcaa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:05.438895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ba75f-606d-4c2d-9778-27d92abcaa88 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:05.439183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ba75f-606d-4c2d-9778-27d92abcaa88 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:05.439459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ba75f-606d-4c2d-9778-27d92abcaa88 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:05.440164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ba75f-606d-4c2d-9778-27d92abcaa88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:09:05.440468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ba75f-606d-4c2d-9778-27d92abcaa88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:05.441849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ba75f-606d-4c2d-9778-27d92abcaa88 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:05.451251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 599/1181] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:09:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:05.457869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cdd88-eb8a-41ee-acb0-2e56d440ce46 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:05.562659 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cdd88-eb8a-41ee-acb0-2e56d440ce46 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:05.598507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3cb1f1-54d9-4144-a5e6-64299d28b3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:05.599143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3cb1f1-54d9-4144-a5e6-64299d28b3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:05.599426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3cb1f1-54d9-4144-a5e6-64299d28b3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:05.599736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3cb1f1-54d9-4144-a5e6-64299d28b3f6 None admin] PATH_INFO: `/v3/users/2ef7a0fc2b0a4c1d99838e0c770aa8c7` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:05.614120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3cb1f1-54d9-4144-a5e6-64299d28b3f6 None admin] RBAC: Authorizing `identity:get_user(user_id=2ef7a0fc2b0a4c1d99838e0c770aa8c7)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:05.615675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3cb1f1-54d9-4144-a5e6-64299d28b3f6 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:05.617432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 583/1182] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:09:05 2022] GET /identity/v3/users/2ef7a0fc2b0a4c1d99838e0c770aa8c7 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:05.623436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ba75f-606d-4c2d-9778-27d92abcaa88 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:05.721886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:05.730809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ba75f-606d-4c2d-9778-27d92abcaa88 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:05.765724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a2d96-29ba-4196-a650-711bcfe47868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:05.766419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a2d96-29ba-4196-a650-711bcfe47868 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:05.766713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a2d96-29ba-4196-a650-711bcfe47868 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:05.766996 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a2d96-29ba-4196-a650-711bcfe47868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:05.840203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a2d96-29ba-4196-a650-711bcfe47868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:05.841563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a2d96-29ba-4196-a650-711bcfe47868 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:05.939849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:05.949469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 600/1183] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:09:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:09:05.955999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3cb1f1-54d9-4144-a5e6-64299d28b3f6 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:06.007118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:06.058030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3cb1f1-54d9-4144-a5e6-64299d28b3f6 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:06.092605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aecdca9-8c6e-4448-a2ff-70cfb1435a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:06.093931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecdca9-8c6e-4448-a2ff-70cfb1435a01 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:06.094210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecdca9-8c6e-4448-a2ff-70cfb1435a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:06.094484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aecdca9-8c6e-4448-a2ff-70cfb1435a01 None admin] PATH_INFO: `/v3/projects/90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:06.095656 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aecdca9-8c6e-4448-a2ff-70cfb1435a01 None admin] RBAC: Authorizing `identity:get_project(project_id=90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:06.097094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aecdca9-8c6e-4448-a2ff-70cfb1435a01 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:06.098779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 584/1184] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:09:05 2022] GET /identity/v3/projects/90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:06.105082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a2d96-29ba-4196-a650-711bcfe47868 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:06.204853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a2d96-29ba-4196-a650-711bcfe47868 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:06.239757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b56f5c-e075-4279-963b-7018d3dc84e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:06.240370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56f5c-e075-4279-963b-7018d3dc84e3 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:06.240692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56f5c-e075-4279-963b-7018d3dc84e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:06.240970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b56f5c-e075-4279-963b-7018d3dc84e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:06.241710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b56f5c-e075-4279-963b-7018d3dc84e3 None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa, user.id=2ef7a0fc2b0a4c1d99838e0c770aa8c7) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:09:06.242020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b56f5c-e075-4279-963b-7018d3dc84e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:06.243482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b56f5c-e075-4279-963b-7018d3dc84e3 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:06.250085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 601/1185] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:09:06 2022] GET /identity/v3/role_assignments?user.id=2ef7a0fc2b0a4c1d99838e0c770aa8c7&scope.project.id=90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:08.431339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cd25c-cd37-4f2d-acc8-c81520790cc6 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:08.432036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cd25c-cd37-4f2d-acc8-c81520790cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:08.432036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cd25c-cd37-4f2d-acc8-c81520790cc6 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:08.433290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 585/1186] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:09:08.439412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1081cfc-07c3-43cc-b36f-04621f3ae1a8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:08.439885 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1081cfc-07c3-43cc-b36f-04621f3ae1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:08.440004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1081cfc-07c3-43cc-b36f-04621f3ae1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:08.459488 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a1081cfc-07c3-43cc-b36f-04621f3ae1a8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:09:08.460724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a1081cfc-07c3-43cc-b36f-04621f3ae1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:09:08.513712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:08.531047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 602/1187] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:09:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:09:08.799633 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295cd25c-cd37-4f2d-acc8-c81520790cc6 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:08.905423 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295cd25c-cd37-4f2d-acc8-c81520790cc6 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:08.923473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:08.942107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e42be-f7f3-404f-9230-3ed904579120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:08.942730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e42be-f7f3-404f-9230-3ed904579120 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:08.943008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e42be-f7f3-404f-9230-3ed904579120 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:08.943329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e42be-f7f3-404f-9230-3ed904579120 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:08.954880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e42be-f7f3-404f-9230-3ed904579120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-c68e42be-f7f3-404f-9230-3ed904579120 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:09:08.960475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:09:08.962568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:09:08.964299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 586/1188] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:09:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:09:08.970685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1081cfc-07c3-43cc-b36f-04621f3ae1a8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:09.076503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1081cfc-07c3-43cc-b36f-04621f3ae1a8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:09.126333 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a941e34-ea94-48be-ad45-7ebc6b3cc4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:09.126953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a941e34-ea94-48be-ad45-7ebc6b3cc4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:09.127230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a941e34-ea94-48be-ad45-7ebc6b3cc4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:09.127500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a941e34-ea94-48be-ad45-7ebc6b3cc4e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:09.128201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a941e34-ea94-48be-ad45-7ebc6b3cc4e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:09:09.128500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a941e34-ea94-48be-ad45-7ebc6b3cc4e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:09.129887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a941e34-ea94-48be-ad45-7ebc6b3cc4e5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:09.139027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 603/1189] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:09:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:09.147803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e42be-f7f3-404f-9230-3ed904579120 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:09.266804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e42be-f7f3-404f-9230-3ed904579120 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:09.303464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad2184-5c40-4a30-ad35-0e5766176882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:09.304133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad2184-5c40-4a30-ad35-0e5766176882 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:09.304416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad2184-5c40-4a30-ad35-0e5766176882 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:09.304690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad2184-5c40-4a30-ad35-0e5766176882 None admin] PATH_INFO: `/v3/users/2ef7a0fc2b0a4c1d99838e0c770aa8c7` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:09.305839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad2184-5c40-4a30-ad35-0e5766176882 None admin] RBAC: Authorizing `identity:get_user(user_id=2ef7a0fc2b0a4c1d99838e0c770aa8c7)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:09.307282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad2184-5c40-4a30-ad35-0e5766176882 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:09.308949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 587/1190] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:09:09 2022] GET /identity/v3/users/2ef7a0fc2b0a4c1d99838e0c770aa8c7 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:09.315490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a941e34-ea94-48be-ad45-7ebc6b3cc4e5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:09.364924 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:09.416106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a941e34-ea94-48be-ad45-7ebc6b3cc4e5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:09.450285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df755f24-a28a-4c7c-9900-433b90013a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:09.450987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755f24-a28a-4c7c-9900-433b90013a9a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:09.451189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755f24-a28a-4c7c-9900-433b90013a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:09.451461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755f24-a28a-4c7c-9900-433b90013a9a None admin] PATH_INFO: `/v3/projects/90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:09.453272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df755f24-a28a-4c7c-9900-433b90013a9a None admin] RBAC: Authorizing `identity:get_project(project_id=90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:09.454726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df755f24-a28a-4c7c-9900-433b90013a9a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:09.456508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 604/1191] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:09:09 2022] GET /identity/v3/projects/90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:09.462757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad2184-5c40-4a30-ad35-0e5766176882 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:09.511492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:09.572512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad2184-5c40-4a30-ad35-0e5766176882 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:09.607707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd7a87-0177-48c0-bf51-bc4883883ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:09.608372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd7a87-0177-48c0-bf51-bc4883883ef0 None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:09.609182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd7a87-0177-48c0-bf51-bc4883883ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:09.609460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd7a87-0177-48c0-bf51-bc4883883ef0 None admin] PATH_INFO: `/v3/projects/90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa/users/2ef7a0fc2b0a4c1d99838e0c770aa8c7/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:09.611402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd7a87-0177-48c0-bf51-bc4883883ef0 None admin] RBAC: Authorizing `identity:create_grant(project_id=90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa, user_id=2ef7a0fc2b0a4c1d99838e0c770aa8c7, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:09.612911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd7a87-0177-48c0-bf51-bc4883883ef0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:09.690033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 588/1192] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:09:09 2022] PUT /identity/v3/projects/90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa/users/2ef7a0fc2b0a4c1d99838e0c770aa8c7/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:09:11.882451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82a75e-a8c0-4074-813e-689e69e9be63 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:11.883016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82a75e-a8c0-4074-813e-689e69e9be63 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:11.883016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f82a75e-a8c0-4074-813e-689e69e9be63 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:11.884617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 605/1193] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:09:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:09:11.890307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd890f-f1f7-4a9a-af73-147e556728f1 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:11.890720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd890f-f1f7-4a9a-af73-147e556728f1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:11.890851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd890f-f1f7-4a9a-af73-147e556728f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:11.909995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-08bd890f-f1f7-4a9a-af73-147e556728f1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:09:11.911216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-08bd890f-f1f7-4a9a-af73-147e556728f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:09:11.980291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 589/1194] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:09:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:09:12.248453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82a75e-a8c0-4074-813e-689e69e9be63 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:12.351373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f82a75e-a8c0-4074-813e-689e69e9be63 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:12.367724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:12.385987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b514146-e4e7-4ebc-9747-bd3f6afedd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:12.386624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b514146-e4e7-4ebc-9747-bd3f6afedd3c None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:12.386908 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b514146-e4e7-4ebc-9747-bd3f6afedd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:12.387184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b514146-e4e7-4ebc-9747-bd3f6afedd3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:12.398662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b514146-e4e7-4ebc-9747-bd3f6afedd3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-4b514146-e4e7-4ebc-9747-bd3f6afedd3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:09:12.403677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:09:12.404858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:09:12.405932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 606/1195] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:09:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:09:12.411043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd890f-f1f7-4a9a-af73-147e556728f1 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:12.444842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:12.511849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd890f-f1f7-4a9a-af73-147e556728f1 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:12.548065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8c466-1a6d-4cd5-885f-fdb3d959e2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:12.548663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8c466-1a6d-4cd5-885f-fdb3d959e2ac None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:12.548939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8c466-1a6d-4cd5-885f-fdb3d959e2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:12.549209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8c466-1a6d-4cd5-885f-fdb3d959e2ac None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:12.549868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8c466-1a6d-4cd5-885f-fdb3d959e2ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:09:12.550166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8c466-1a6d-4cd5-885f-fdb3d959e2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:12.551525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8c466-1a6d-4cd5-885f-fdb3d959e2ac None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:12.559735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 590/1196] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:09:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:12.566959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b514146-e4e7-4ebc-9747-bd3f6afedd3c None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:12.675717 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b514146-e4e7-4ebc-9747-bd3f6afedd3c None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:12.715748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8677-b11c-4b19-85c7-1c1e2838efc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:12.716400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8677-b11c-4b19-85c7-1c1e2838efc5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:12.716685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8677-b11c-4b19-85c7-1c1e2838efc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:12.716964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b8677-b11c-4b19-85c7-1c1e2838efc5 None admin] PATH_INFO: `/v3/users/2ef7a0fc2b0a4c1d99838e0c770aa8c7` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:12.718131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b8677-b11c-4b19-85c7-1c1e2838efc5 None admin] RBAC: Authorizing `identity:get_user(user_id=2ef7a0fc2b0a4c1d99838e0c770aa8c7)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:12.719587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b8677-b11c-4b19-85c7-1c1e2838efc5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:12.721236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 607/1197] 10.222.0.64 () {56 vars in 1177 bytes} [Fri Sep 23 01:09:12 2022] GET /identity/v3/users/2ef7a0fc2b0a4c1d99838e0c770aa8c7 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:12.729231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8c466-1a6d-4cd5-885f-fdb3d959e2ac None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:12.825738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:12.832034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8c466-1a6d-4cd5-885f-fdb3d959e2ac None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:12.866354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1e72-a102-42ab-9843-f25402a2c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:12.866981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1e72-a102-42ab-9843-f25402a2c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:12.867238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1e72-a102-42ab-9843-f25402a2c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:12.867508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d1e72-a102-42ab-9843-f25402a2c3c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:12.940511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d1e72-a102-42ab-9843-f25402a2c3c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:12.941992 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d1e72-a102-42ab-9843-f25402a2c3c6 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:13.038799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:13.045534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 591/1198] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:09:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:09:13.051584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8677-b11c-4b19-85c7-1c1e2838efc5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:13.100529 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:13.156166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b8677-b11c-4b19-85c7-1c1e2838efc5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:13.190509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b3d5e-05e6-426e-adf5-dfc5ff6f11fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:13.191149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b3d5e-05e6-426e-adf5-dfc5ff6f11fc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:13.191429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b3d5e-05e6-426e-adf5-dfc5ff6f11fc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:13.191747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b3d5e-05e6-426e-adf5-dfc5ff6f11fc None admin] PATH_INFO: `/v3/projects/90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:13.192885 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b3d5e-05e6-426e-adf5-dfc5ff6f11fc None admin] RBAC: Authorizing `identity:get_project(project_id=90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:13.194323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b3d5e-05e6-426e-adf5-dfc5ff6f11fc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:13.196126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 608/1199] 10.222.0.64 () {56 vars in 1186 bytes} [Fri Sep 23 01:09:13 2022] GET /identity/v3/projects/90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:13.203252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1e72-a102-42ab-9843-f25402a2c3c6 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:09:13.303346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d1e72-a102-42ab-9843-f25402a2c3c6 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:09:13.338636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cb0c4-a962-4df6-9223-4478c2a833ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:09:13.339236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cb0c4-a962-4df6-9223-4478c2a833ae None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:09:13.339515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cb0c4-a962-4df6-9223-4478c2a833ae None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:09:13.339824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cb0c4-a962-4df6-9223-4478c2a833ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:09:13.340553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cb0c4-a962-4df6-9223-4478c2a833ae None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa, user.id=2ef7a0fc2b0a4c1d99838e0c770aa8c7) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:09:13.340864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cb0c4-a962-4df6-9223-4478c2a833ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:09:13.342287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cb0c4-a962-4df6-9223-4478c2a833ae None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:09:13.348204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 592/1200] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:09:13 2022] GET /identity/v3/role_assignments?user.id=2ef7a0fc2b0a4c1d99838e0c770aa8c7&scope.project.id=90d5f6ef4f2a4a2c9a0c0fcc7ed6bafa&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:09:23.185588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=44 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:09:23.345596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=70 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:43.435809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fec87-7af2-4f73-bd53-548882208ed3 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:43.436442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fec87-7af2-4f73-bd53-548882208ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:43.436442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fec87-7af2-4f73-bd53-548882208ed3 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:43.438362 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 609/1201] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:10:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:10:43.444209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e516-16ab-49ec-846b-7da38ce03872 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:43.444759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e516-16ab-49ec-846b-7da38ce03872 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:43.444759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e516-16ab-49ec-846b-7da38ce03872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:43.465295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ea52e516-16ab-49ec-846b-7da38ce03872 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:10:43.472310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ea52e516-16ab-49ec-846b-7da38ce03872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:10:43.540765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 593/1202] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:10:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:10:43.814321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fec87-7af2-4f73-bd53-548882208ed3 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:43.954324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fec87-7af2-4f73-bd53-548882208ed3 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:43.997792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a68cb-ff9d-4f79-82cd-f80ab3c6f5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:43.998424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a68cb-ff9d-4f79-82cd-f80ab3c6f5ed None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:43.998702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a68cb-ff9d-4f79-82cd-f80ab3c6f5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:43.998973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a68cb-ff9d-4f79-82cd-f80ab3c6f5ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:44.010260 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a68cb-ff9d-4f79-82cd-f80ab3c6f5ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:44.011738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a68cb-ff9d-4f79-82cd-f80ab3c6f5ed None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-7b2a68cb-ff9d-4f79-82cd-f80ab3c6f5ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:10:44.030338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:44.038095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:10:44.039104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:10:44.039104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:44.039104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:10:44.039104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 610/1203] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:44.039408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e516-16ab-49ec-846b-7da38ce03872 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:44.158972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e516-16ab-49ec-846b-7da38ce03872 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:44.203889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:44.213403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f6921-c431-4de2-a0dd-5ab519022297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:44.214437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6921-c431-4de2-a0dd-5ab519022297 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:44.214913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6921-c431-4de2-a0dd-5ab519022297 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:44.215355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f6921-c431-4de2-a0dd-5ab519022297 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:44.216539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f6921-c431-4de2-a0dd-5ab519022297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:44.217047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f6921-c431-4de2-a0dd-5ab519022297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:44.220420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f6921-c431-4de2-a0dd-5ab519022297 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:44.242896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 594/1204] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:44.252563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a68cb-ff9d-4f79-82cd-f80ab3c6f5ed None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:44.351096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:44.432790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a68cb-ff9d-4f79-82cd-f80ab3c6f5ed None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:44.476081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbd708-bf75-4692-8eb1-4a325fe23b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:44.476795 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbd708-bf75-4692-8eb1-4a325fe23b32 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:44.476976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbd708-bf75-4692-8eb1-4a325fe23b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:44.477259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbd708-bf75-4692-8eb1-4a325fe23b32 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:44.477919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbd708-bf75-4692-8eb1-4a325fe23b32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:44.479400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbd708-bf75-4692-8eb1-4a325fe23b32 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:44.644658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 611/1205] 10.222.0.64 () {60 vars in 1132 bytes} [Fri Sep 23 01:10:44 2022] POST /identity/v3/users => generated 251 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:10:46.909096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475084c-90dc-4066-b05c-9a347039b28f None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:46.909858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475084c-90dc-4066-b05c-9a347039b28f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:46.910296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c475084c-90dc-4066-b05c-9a347039b28f None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:46.912924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 595/1206] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:10:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:10:46.919706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f4c93-5878-4ff2-b03a-ebcdee1dba18 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:46.920311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f4c93-5878-4ff2-b03a-ebcdee1dba18 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:46.920311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f4c93-5878-4ff2-b03a-ebcdee1dba18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:46.942135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-009f4c93-5878-4ff2-b03a-ebcdee1dba18 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:10:46.943335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-009f4c93-5878-4ff2-b03a-ebcdee1dba18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:10:47.030928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 612/1207] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:10:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 23 01:10:47.306495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475084c-90dc-4066-b05c-9a347039b28f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:47.456217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c475084c-90dc-4066-b05c-9a347039b28f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:47.497623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b3aef-3424-4a3a-99be-f36e66972a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:47.498336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b3aef-3424-4a3a-99be-f36e66972a94 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:47.498618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b3aef-3424-4a3a-99be-f36e66972a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:47.498888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b3aef-3424-4a3a-99be-f36e66972a94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:47.511671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b3aef-3424-4a3a-99be-f36e66972a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-f65b3aef-3424-4a3a-99be-f36e66972a94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:10:47.516871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:47.521073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:10:47.522177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 596/1208] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:10:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:47.524732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f4c93-5878-4ff2-b03a-ebcdee1dba18 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:47.571683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:47.636400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f4c93-5878-4ff2-b03a-ebcdee1dba18 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:47.697236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341be462-5610-438f-8037-7992db1c9067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:47.698348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341be462-5610-438f-8037-7992db1c9067 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:47.698846 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341be462-5610-438f-8037-7992db1c9067 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:47.699628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341be462-5610-438f-8037-7992db1c9067 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:47.700936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341be462-5610-438f-8037-7992db1c9067 None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:47.701496 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341be462-5610-438f-8037-7992db1c9067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:47.704280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341be462-5610-438f-8037-7992db1c9067 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:47.723113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 613/1209] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:10:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:47.729120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b3aef-3424-4a3a-99be-f36e66972a94 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:47.786487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:47.846809 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b3aef-3424-4a3a-99be-f36e66972a94 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:47.887563 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faa044-2f52-4ba5-a76d-3bd92ced7982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:47.888184 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faa044-2f52-4ba5-a76d-3bd92ced7982 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:47.888447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faa044-2f52-4ba5-a76d-3bd92ced7982 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:47.888656 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28faa044-2f52-4ba5-a76d-3bd92ced7982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:47.900836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faa044-2f52-4ba5-a76d-3bd92ced7982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:47.902450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28faa044-2f52-4ba5-a76d-3bd92ced7982 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-28faa044-2f52-4ba5-a76d-3bd92ced7982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:10:47.918439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:47.920174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:10:47.921201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:10:47.921201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:47.921201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:10:47.921201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 597/1210] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:47.926454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341be462-5610-438f-8037-7992db1c9067 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:48.086904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:48.091656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341be462-5610-438f-8037-7992db1c9067 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:48.141463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c6339-3d4f-46be-b3d8-50bc0943b79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:48.142131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c6339-3d4f-46be-b3d8-50bc0943b79f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:48.142411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c6339-3d4f-46be-b3d8-50bc0943b79f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:48.142694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c6339-3d4f-46be-b3d8-50bc0943b79f None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:48.143344 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c6339-3d4f-46be-b3d8-50bc0943b79f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:48.143673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c6339-3d4f-46be-b3d8-50bc0943b79f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:48.145058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c6339-3d4f-46be-b3d8-50bc0943b79f None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:48.158779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 614/1211] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:48.165674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faa044-2f52-4ba5-a76d-3bd92ced7982 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:48.307204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28faa044-2f52-4ba5-a76d-3bd92ced7982 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:48.308902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:48.346429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196ad2e-3fac-4a5d-8c03-cf4af4cdb498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:48.346972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ad2e-3fac-4a5d-8c03-cf4af4cdb498 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:48.347247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ad2e-3fac-4a5d-8c03-cf4af4cdb498 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:48.347543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ad2e-3fac-4a5d-8c03-cf4af4cdb498 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:48.356285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196ad2e-3fac-4a5d-8c03-cf4af4cdb498 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:48.358594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196ad2e-3fac-4a5d-8c03-cf4af4cdb498 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-6196ad2e-3fac-4a5d-8c03-cf4af4cdb498 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:10:48.371109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:48.372579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:10:48.373868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 598/1212] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:10:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:48.378382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c6339-3d4f-46be-b3d8-50bc0943b79f None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:48.494102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c6339-3d4f-46be-b3d8-50bc0943b79f None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:48.538777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a169850-be31-4111-9e7c-6a0c435cdfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:48.539826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169850-be31-4111-9e7c-6a0c435cdfd5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:48.540114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169850-be31-4111-9e7c-6a0c435cdfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:48.540404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a169850-be31-4111-9e7c-6a0c435cdfd5 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:48.541213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a169850-be31-4111-9e7c-6a0c435cdfd5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:48.541627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a169850-be31-4111-9e7c-6a0c435cdfd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:48.543057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a169850-be31-4111-9e7c-6a0c435cdfd5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:48.563650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 615/1213] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Sep 23 01:10:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:48.571213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196ad2e-3fac-4a5d-8c03-cf4af4cdb498 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:48.703459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196ad2e-3fac-4a5d-8c03-cf4af4cdb498 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:48.742077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7e0fc-292d-479d-b0c6-2ed0b09a5736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:48.742698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7e0fc-292d-479d-b0c6-2ed0b09a5736 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:48.742980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7e0fc-292d-479d-b0c6-2ed0b09a5736 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:48.743254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7e0fc-292d-479d-b0c6-2ed0b09a5736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:48.778000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:48.824703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7e0fc-292d-479d-b0c6-2ed0b09a5736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:48.826276 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7e0fc-292d-479d-b0c6-2ed0b09a5736 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:48.934569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 599/1214] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:10:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:10:48.940517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a169850-be31-4111-9e7c-6a0c435cdfd5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:48.962884 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:49.047626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a169850-be31-4111-9e7c-6a0c435cdfd5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:49.082311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1c139-3fbf-41a6-841b-2b6ede516de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:49.082952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1c139-3fbf-41a6-841b-2b6ede516de5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:49.083236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1c139-3fbf-41a6-841b-2b6ede516de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:49.083512 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1c139-3fbf-41a6-841b-2b6ede516de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:49.094236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1c139-3fbf-41a6-841b-2b6ede516de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:49.095722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e1c139-3fbf-41a6-841b-2b6ede516de5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-07e1c139-3fbf-41a6-841b-2b6ede516de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:10:49.111029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:49.117031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:10:49.118235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:10:49.118235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:49.118235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:10:49.118235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 616/1215] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:49.118609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7e0fc-292d-479d-b0c6-2ed0b09a5736 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:49.188806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:49.231538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7e0fc-292d-479d-b0c6-2ed0b09a5736 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:49.268986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42eb9b9-dd5d-4f43-84c4-00c9a755f0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:49.270592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42eb9b9-dd5d-4f43-84c4-00c9a755f0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:49.271154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42eb9b9-dd5d-4f43-84c4-00c9a755f0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:49.271568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42eb9b9-dd5d-4f43-84c4-00c9a755f0a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:49.272592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42eb9b9-dd5d-4f43-84c4-00c9a755f0a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:49.273020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42eb9b9-dd5d-4f43-84c4-00c9a755f0a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:49.274794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42eb9b9-dd5d-4f43-84c4-00c9a755f0a9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:49.289347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 600/1216] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:49.298535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1c139-3fbf-41a6-841b-2b6ede516de5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:49.371883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:49.404703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e1c139-3fbf-41a6-841b-2b6ede516de5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:49.438789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05976f6-6ebe-4c59-bebd-bc1d22af871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:49.439469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05976f6-6ebe-4c59-bebd-bc1d22af871d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:49.439768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05976f6-6ebe-4c59-bebd-bc1d22af871d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:49.440039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05976f6-6ebe-4c59-bebd-bc1d22af871d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:49.447130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05976f6-6ebe-4c59-bebd-bc1d22af871d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:49.448639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05976f6-6ebe-4c59-bebd-bc1d22af871d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-b05976f6-6ebe-4c59-bebd-bc1d22af871d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:10:49.461044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:49.467909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:49.469555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:10:49.469555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:10:49.469555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:10:49.469555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:10:49.469555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:10:49.469555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:10:49.469555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:10:49.469555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:10:49.469555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 617/1217] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:49.470176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42eb9b9-dd5d-4f43-84c4-00c9a755f0a9 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:49.573526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42eb9b9-dd5d-4f43-84c4-00c9a755f0a9 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:49.598853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:49.610836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab5ad6-f6eb-4fe8-b9f3-9e41a9a4f266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:49.611451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab5ad6-f6eb-4fe8-b9f3-9e41a9a4f266 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:49.611755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab5ad6-f6eb-4fe8-b9f3-9e41a9a4f266 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:49.612030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab5ad6-f6eb-4fe8-b9f3-9e41a9a4f266 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:49.612798 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab5ad6-f6eb-4fe8-b9f3-9e41a9a4f266 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:49.613109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab5ad6-f6eb-4fe8-b9f3-9e41a9a4f266 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:49.614723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab5ad6-f6eb-4fe8-b9f3-9e41a9a4f266 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:49.626570 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 601/1218] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:10:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:49.632927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05976f6-6ebe-4c59-bebd-bc1d22af871d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:49.742244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05976f6-6ebe-4c59-bebd-bc1d22af871d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:49.772723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:49.779182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b971f7a-9349-4152-904f-5b4344ff2c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:49.779844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b971f7a-9349-4152-904f-5b4344ff2c8a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:49.780123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b971f7a-9349-4152-904f-5b4344ff2c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:49.780397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b971f7a-9349-4152-904f-5b4344ff2c8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:49.781201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b971f7a-9349-4152-904f-5b4344ff2c8a None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=9f73bca3896440258430cfce042faa42) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:49.781510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b971f7a-9349-4152-904f-5b4344ff2c8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:49.783042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b971f7a-9349-4152-904f-5b4344ff2c8a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:49.789032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 618/1219] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:10:49 2022] GET /identity/v3/role_assignments?user.id=9f73bca3896440258430cfce042faa42&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:52.007810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552e52b-7a62-4308-b26d-a1560170551f None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:52.008353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552e52b-7a62-4308-b26d-a1560170551f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:52.008451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552e52b-7a62-4308-b26d-a1560170551f None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:52.009845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 602/1220] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:10:52.015455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da46aec-0cb4-43fb-9a06-0bb21d1207fa None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:52.016787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da46aec-0cb4-43fb-9a06-0bb21d1207fa None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:52.016787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da46aec-0cb4-43fb-9a06-0bb21d1207fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:52.036872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9da46aec-0cb4-43fb-9a06-0bb21d1207fa None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:10:52.038123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9da46aec-0cb4-43fb-9a06-0bb21d1207fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:10:52.115235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 619/1221] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:10:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:10:52.385382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552e52b-7a62-4308-b26d-a1560170551f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:52.507540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552e52b-7a62-4308-b26d-a1560170551f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:52.547789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8221c9-06d6-41ac-a391-3119641ff0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:52.548436 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8221c9-06d6-41ac-a391-3119641ff0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:52.548714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8221c9-06d6-41ac-a391-3119641ff0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:52.548982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8221c9-06d6-41ac-a391-3119641ff0a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:52.561183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8221c9-06d6-41ac-a391-3119641ff0a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-5c8221c9-06d6-41ac-a391-3119641ff0a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:10:52.566019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:52.567669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:10:52.568720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 603/1222] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:10:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:52.574102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da46aec-0cb4-43fb-9a06-0bb21d1207fa None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:52.699713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da46aec-0cb4-43fb-9a06-0bb21d1207fa None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:52.726280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:52.737430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53fe20-4e04-42ef-b65f-80bf7f16ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:52.738065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53fe20-4e04-42ef-b65f-80bf7f16ef92 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:52.738390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53fe20-4e04-42ef-b65f-80bf7f16ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:52.738864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53fe20-4e04-42ef-b65f-80bf7f16ef92 None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:52.739663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53fe20-4e04-42ef-b65f-80bf7f16ef92 None admin] RBAC: Adding query filter params (name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:52.739975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53fe20-4e04-42ef-b65f-80bf7f16ef92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:52.741377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af53fe20-4e04-42ef-b65f-80bf7f16ef92 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:52.751028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 620/1223] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:10:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:10:52.757384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8221c9-06d6-41ac-a391-3119641ff0a4 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:52.802878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:52.872527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8221c9-06d6-41ac-a391-3119641ff0a4 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:52.914560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e405ef-d827-46a7-bc17-f718db92bd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:52.915213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e405ef-d827-46a7-bc17-f718db92bd07 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:52.915488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e405ef-d827-46a7-bc17-f718db92bd07 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:52.915806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e405ef-d827-46a7-bc17-f718db92bd07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:52.927894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e405ef-d827-46a7-bc17-f718db92bd07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:52.929366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e405ef-d827-46a7-bc17-f718db92bd07 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-15e405ef-d827-46a7-bc17-f718db92bd07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:10:52.947159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:52.948460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:10:52.949639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:10:52.949639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:52.949639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:10:52.949639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 604/1224] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:52.957923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53fe20-4e04-42ef-b65f-80bf7f16ef92 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:53.070780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53fe20-4e04-42ef-b65f-80bf7f16ef92 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:53.109909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5a101-6e3d-4c66-a3d5-208c504775b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:53.110540 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5a101-6e3d-4c66-a3d5-208c504775b1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:53.110817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5a101-6e3d-4c66-a3d5-208c504775b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:53.111090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5a101-6e3d-4c66-a3d5-208c504775b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:53.111784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5a101-6e3d-4c66-a3d5-208c504775b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:53.112088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5a101-6e3d-4c66-a3d5-208c504775b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:53.113513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5a101-6e3d-4c66-a3d5-208c504775b1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:53.127425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 621/1225] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:53.133621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e405ef-d827-46a7-bc17-f718db92bd07 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:53.239612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:53.248364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e405ef-d827-46a7-bc17-f718db92bd07 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:53.290398 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72766f5c-4eb4-4b50-be9d-4e539e0727fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:53.291077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72766f5c-4eb4-4b50-be9d-4e539e0727fc None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:53.291394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72766f5c-4eb4-4b50-be9d-4e539e0727fc None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:53.291715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72766f5c-4eb4-4b50-be9d-4e539e0727fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:53.301496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72766f5c-4eb4-4b50-be9d-4e539e0727fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:53.302977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72766f5c-4eb4-4b50-be9d-4e539e0727fc None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-72766f5c-4eb4-4b50-be9d-4e539e0727fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:10:53.317525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:53.318982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:10:53.320326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 605/1226] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:10:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:53.325426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5a101-6e3d-4c66-a3d5-208c504775b1 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:53.385461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:53.444032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5a101-6e3d-4c66-a3d5-208c504775b1 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:53.485719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75281043-0f29-4ef4-926e-db1150a218ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:53.486324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75281043-0f29-4ef4-926e-db1150a218ba None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:53.486604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75281043-0f29-4ef4-926e-db1150a218ba None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:53.486881 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75281043-0f29-4ef4-926e-db1150a218ba None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:53.487645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75281043-0f29-4ef4-926e-db1150a218ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:53.487956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75281043-0f29-4ef4-926e-db1150a218ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:53.489383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75281043-0f29-4ef4-926e-db1150a218ba None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:53.516357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 622/1227] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Sep 23 01:10:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:53.516584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72766f5c-4eb4-4b50-be9d-4e539e0727fc None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:53.617779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72766f5c-4eb4-4b50-be9d-4e539e0727fc None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:53.654850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c357569-6d9e-4ec3-9374-80f54bb5b187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:53.655515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c357569-6d9e-4ec3-9374-80f54bb5b187 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:53.655831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c357569-6d9e-4ec3-9374-80f54bb5b187 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:53.656628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c357569-6d9e-4ec3-9374-80f54bb5b187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:53.669184 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c357569-6d9e-4ec3-9374-80f54bb5b187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:53.670653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c357569-6d9e-4ec3-9374-80f54bb5b187 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-3c357569-6d9e-4ec3-9374-80f54bb5b187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:10:53.685885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:53.688201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:10:53.690163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:10:53.690163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:53.690163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:10:53.690163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 606/1228] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:53.693008 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75281043-0f29-4ef4-926e-db1150a218ba None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:53.799734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75281043-0f29-4ef4-926e-db1150a218ba None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:53.802207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:53.836662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a663a9-4715-4f7a-a0ca-adfdc1a233f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:53.837313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a663a9-4715-4f7a-a0ca-adfdc1a233f8 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:53.837598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a663a9-4715-4f7a-a0ca-adfdc1a233f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:53.837875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a663a9-4715-4f7a-a0ca-adfdc1a233f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:53.838535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a663a9-4715-4f7a-a0ca-adfdc1a233f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:53.838841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a663a9-4715-4f7a-a0ca-adfdc1a233f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:53.840255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a663a9-4715-4f7a-a0ca-adfdc1a233f8 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:53.857120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 623/1229] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:53.858746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c357569-6d9e-4ec3-9374-80f54bb5b187 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:53.868689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:53.965860 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c357569-6d9e-4ec3-9374-80f54bb5b187 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:54.001017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e73f3-db9a-4c2c-9c2e-03c17beae97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:54.001655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e73f3-db9a-4c2c-9c2e-03c17beae97c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:54.001931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e73f3-db9a-4c2c-9c2e-03c17beae97c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:54.002206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e73f3-db9a-4c2c-9c2e-03c17beae97c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:54.009027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e73f3-db9a-4c2c-9c2e-03c17beae97c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:54.010481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e73f3-db9a-4c2c-9c2e-03c17beae97c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-413e73f3-db9a-4c2c-9c2e-03c17beae97c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:10:54.022333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:54.023447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:54.024485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:10:54.024485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:10:54.024485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:10:54.024485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:10:54.024485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:10:54.024485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:10:54.024485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:10:54.024485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:10:54.024485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 607/1230] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:10:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:54.030428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a663a9-4715-4f7a-a0ca-adfdc1a233f8 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:54.149640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a663a9-4715-4f7a-a0ca-adfdc1a233f8 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:54.186636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf8481-2f71-48a4-adb1-84844f2027b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:54.187605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf8481-2f71-48a4-adb1-84844f2027b6 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:54.187891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf8481-2f71-48a4-adb1-84844f2027b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:54.188164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf8481-2f71-48a4-adb1-84844f2027b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:54.189297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf8481-2f71-48a4-adb1-84844f2027b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:54.189297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf8481-2f71-48a4-adb1-84844f2027b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:54.190606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf8481-2f71-48a4-adb1-84844f2027b6 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:54.204490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 624/1231] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:10:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:54.210879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e73f3-db9a-4c2c-9c2e-03c17beae97c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:54.281063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:54.321290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e73f3-db9a-4c2c-9c2e-03c17beae97c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:54.356476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5fd44-d851-4ea2-b982-4bca04eadc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:54.357226 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5fd44-d851-4ea2-b982-4bca04eadc96 None admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:54.357416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5fd44-d851-4ea2-b982-4bca04eadc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:54.357690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5fd44-d851-4ea2-b982-4bca04eadc96 None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/9f73bca3896440258430cfce042faa42/roles/50ceaff2140f4e05a56877fee2f609e4` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:54.373299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5fd44-d851-4ea2-b982-4bca04eadc96 None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=9f73bca3896440258430cfce042faa42, role_id=50ceaff2140f4e05a56877fee2f609e4)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:54.375259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5fd44-d851-4ea2-b982-4bca04eadc96 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:54.495204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 608/1232] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:10:54 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/9f73bca3896440258430cfce042faa42/roles/50ceaff2140f4e05a56877fee2f609e4 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:10:56.690818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9e06f-8ec0-4450-912f-71632df0303b None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:56.691444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9e06f-8ec0-4450-912f-71632df0303b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:56.691444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9e06f-8ec0-4450-912f-71632df0303b None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:56.692816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 625/1233] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:10:56.698713 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0193f-9bb9-46b1-ac7a-568704dc62e2 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:56.699218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0193f-9bb9-46b1-ac7a-568704dc62e2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:56.699376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0193f-9bb9-46b1-ac7a-568704dc62e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:56.718367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-18c0193f-9bb9-46b1-ac7a-568704dc62e2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:10:56.719599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-18c0193f-9bb9-46b1-ac7a-568704dc62e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:10:56.800742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 609/1234] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:10:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:10:57.072626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9e06f-8ec0-4450-912f-71632df0303b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:57.109794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:57.198896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9e06f-8ec0-4450-912f-71632df0303b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:57.235178 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b1067-a7ea-4fd7-9b25-aa2d6d839407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:57.235834 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b1067-a7ea-4fd7-9b25-aa2d6d839407 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:57.236113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b1067-a7ea-4fd7-9b25-aa2d6d839407 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:57.236384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b1067-a7ea-4fd7-9b25-aa2d6d839407 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:57.254309 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b1067-a7ea-4fd7-9b25-aa2d6d839407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-813b1067-a7ea-4fd7-9b25-aa2d6d839407 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:10:57.259268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:57.269282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:10:57.270268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 626/1235] 10.222.0.64 () {56 vars in 1102 bytes} [Fri Sep 23 01:10:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:57.271094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0193f-9bb9-46b1-ac7a-568704dc62e2 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:57.319380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:57.382303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0193f-9bb9-46b1-ac7a-568704dc62e2 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:57.417014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f5c6c-a6fe-4fe3-a71a-166f9955905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:57.418401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f5c6c-a6fe-4fe3-a71a-166f9955905c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:57.418401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f5c6c-a6fe-4fe3-a71a-166f9955905c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:57.418401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f5c6c-a6fe-4fe3-a71a-166f9955905c None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:57.418797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f5c6c-a6fe-4fe3-a71a-166f9955905c None admin] RBAC: Adding query filter params (name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:57.419094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f5c6c-a6fe-4fe3-a71a-166f9955905c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:57.420499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f5c6c-a6fe-4fe3-a71a-166f9955905c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:57.428750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 610/1236] 10.222.0.64 () {56 vars in 1103 bytes} [Fri Sep 23 01:10:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:57.434922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b1067-a7ea-4fd7-9b25-aa2d6d839407 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:57.527381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:57.544406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b1067-a7ea-4fd7-9b25-aa2d6d839407 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:57.581089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12909cb-6357-4f3d-b179-29b0c70c29fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:57.581741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12909cb-6357-4f3d-b179-29b0c70c29fe None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:57.582072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12909cb-6357-4f3d-b179-29b0c70c29fe None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:57.582429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12909cb-6357-4f3d-b179-29b0c70c29fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:57.596404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12909cb-6357-4f3d-b179-29b0c70c29fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:57.597911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12909cb-6357-4f3d-b179-29b0c70c29fe None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-e12909cb-6357-4f3d-b179-29b0c70c29fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:10:57.615460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:57.617093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:10:57.618565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:10:57.618565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:57.618565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:10:57.618565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 627/1237] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:57.623502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f5c6c-a6fe-4fe3-a71a-166f9955905c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:57.738704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f5c6c-a6fe-4fe3-a71a-166f9955905c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:57.774206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fcc73-f42d-45e5-8264-66a6a31912f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:57.774814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fcc73-f42d-45e5-8264-66a6a31912f0 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:57.775096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fcc73-f42d-45e5-8264-66a6a31912f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:57.775370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fcc73-f42d-45e5-8264-66a6a31912f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:57.776052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fcc73-f42d-45e5-8264-66a6a31912f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:57.776356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fcc73-f42d-45e5-8264-66a6a31912f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:57.777734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fcc73-f42d-45e5-8264-66a6a31912f0 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:57.789950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 611/1238] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:57.795858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12909cb-6357-4f3d-b179-29b0c70c29fe None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:57.845411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:57.916080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12909cb-6357-4f3d-b179-29b0c70c29fe None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:57.950711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d567b-748f-4ac7-aeff-809ce26d4d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:57.951453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d567b-748f-4ac7-aeff-809ce26d4d3b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:57.951763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d567b-748f-4ac7-aeff-809ce26d4d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:57.952038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d567b-748f-4ac7-aeff-809ce26d4d3b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:57.955612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:57.961254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d567b-748f-4ac7-aeff-809ce26d4d3b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:57.964160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d567b-748f-4ac7-aeff-809ce26d4d3b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-8e7d567b-748f-4ac7-aeff-809ce26d4d3b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:10:57.975868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:57.977256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:10:57.978349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 628/1239] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:10:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:57.983813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fcc73-f42d-45e5-8264-66a6a31912f0 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:58.110592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fcc73-f42d-45e5-8264-66a6a31912f0 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:58.161108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e6924-dba0-4a10-aaf9-b98f9a30a48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:58.161723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e6924-dba0-4a10-aaf9-b98f9a30a48d None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:58.162083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e6924-dba0-4a10-aaf9-b98f9a30a48d None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:58.162361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e6924-dba0-4a10-aaf9-b98f9a30a48d None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:58.163106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e6924-dba0-4a10-aaf9-b98f9a30a48d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:58.163416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e6924-dba0-4a10-aaf9-b98f9a30a48d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:58.164897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e6924-dba0-4a10-aaf9-b98f9a30a48d None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:58.183865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 612/1240] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Sep 23 01:10:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:58.190423 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d567b-748f-4ac7-aeff-809ce26d4d3b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:58.297746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d567b-748f-4ac7-aeff-809ce26d4d3b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:58.339417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbc930-c711-46e2-a3db-ffcdfd624509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:58.340325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbc930-c711-46e2-a3db-ffcdfd624509 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:58.340325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbc930-c711-46e2-a3db-ffcdfd624509 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:58.340611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbc930-c711-46e2-a3db-ffcdfd624509 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:58.406160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:58.413226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cbc930-c711-46e2-a3db-ffcdfd624509 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:58.414676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cbc930-c711-46e2-a3db-ffcdfd624509 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:58.557788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 629/1241] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:10:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:10:58.566182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e6924-dba0-4a10-aaf9-b98f9a30a48d None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:58.599237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:58.678051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e6924-dba0-4a10-aaf9-b98f9a30a48d None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:58.727514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463a770-f767-4dbf-9e1c-07c48070e000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:58.728634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463a770-f767-4dbf-9e1c-07c48070e000 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:58.729163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463a770-f767-4dbf-9e1c-07c48070e000 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:58.729637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463a770-f767-4dbf-9e1c-07c48070e000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:58.747788 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463a770-f767-4dbf-9e1c-07c48070e000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:58.750607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463a770-f767-4dbf-9e1c-07c48070e000 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-0463a770-f767-4dbf-9e1c-07c48070e000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:10:58.778110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:58.779375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:10:58.780567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:10:58.780567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:10:58.780567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:10:58.783747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 613/1242] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:58.786194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbc930-c711-46e2-a3db-ffcdfd624509 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:58.897588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:58.906999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cbc930-c711-46e2-a3db-ffcdfd624509 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:58.958544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69059b3f-8567-40e8-afe1-60721962f5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:58.959080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69059b3f-8567-40e8-afe1-60721962f5ce None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:58.959364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69059b3f-8567-40e8-afe1-60721962f5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:58.959671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69059b3f-8567-40e8-afe1-60721962f5ce None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:58.960430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69059b3f-8567-40e8-afe1-60721962f5ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:58.960643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69059b3f-8567-40e8-afe1-60721962f5ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:58.962118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69059b3f-8567-40e8-afe1-60721962f5ce None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:58.975410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 630/1243] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:58.981602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463a770-f767-4dbf-9e1c-07c48070e000 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:59.068877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:10:59.094371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463a770-f767-4dbf-9e1c-07c48070e000 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:59.133234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62117b24-9971-4c88-8a7b-af60fd96ebd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:59.133920 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117b24-9971-4c88-8a7b-af60fd96ebd9 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:59.134199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117b24-9971-4c88-8a7b-af60fd96ebd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:59.134472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117b24-9971-4c88-8a7b-af60fd96ebd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:59.142766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62117b24-9971-4c88-8a7b-af60fd96ebd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:59.144288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62117b24-9971-4c88-8a7b-af60fd96ebd9 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-62117b24-9971-4c88-8a7b-af60fd96ebd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:10:59.157484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:10:59.158740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:10:59.160159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:10:59.160159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:10:59.160159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:10:59.160159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:10:59.160159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:10:59.160159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:10:59.160159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:10:59.160159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:10:59.160159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 614/1244] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:10:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:10:59.165024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69059b3f-8567-40e8-afe1-60721962f5ce None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:59.304645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69059b3f-8567-40e8-afe1-60721962f5ce None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:59.347807 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46ab39-5e5a-4c1a-bab3-02da68375096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:59.348534 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46ab39-5e5a-4c1a-bab3-02da68375096 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:59.348824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46ab39-5e5a-4c1a-bab3-02da68375096 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:59.349092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46ab39-5e5a-4c1a-bab3-02da68375096 None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:59.349799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46ab39-5e5a-4c1a-bab3-02da68375096 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:59.350103 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46ab39-5e5a-4c1a-bab3-02da68375096 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:59.351634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46ab39-5e5a-4c1a-bab3-02da68375096 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:59.364951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 631/1245] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:10:59.371390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62117b24-9971-4c88-8a7b-af60fd96ebd9 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:10:59.477327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62117b24-9971-4c88-8a7b-af60fd96ebd9 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:10:59.513961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597afe80-5245-4697-a9a8-5e25aefd441b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:10:59.514558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597afe80-5245-4697-a9a8-5e25aefd441b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:10:59.514832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597afe80-5245-4697-a9a8-5e25aefd441b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:10:59.515101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597afe80-5245-4697-a9a8-5e25aefd441b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:10:59.515994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597afe80-5245-4697-a9a8-5e25aefd441b None admin] RBAC: Adding query filter params (role.id=50ceaff2140f4e05a56877fee2f609e4, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=9f73bca3896440258430cfce042faa42) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:10:59.521970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597afe80-5245-4697-a9a8-5e25aefd441b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:10:59.521970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597afe80-5245-4697-a9a8-5e25aefd441b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:10:59.524661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 615/1246] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:10:59 2022] GET /identity/v3/role_assignments?user.id=9f73bca3896440258430cfce042faa42&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=50ceaff2140f4e05a56877fee2f609e4 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:11:01.841234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381624-71fc-4421-9c57-b571f57740e6 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:01.845550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381624-71fc-4421-9c57-b571f57740e6 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:01.845550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35381624-71fc-4421-9c57-b571f57740e6 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:01.845550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 632/1247] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:11:01.850061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295fbef-42ec-4f0e-92a3-8d95fd31154b None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:01.850575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295fbef-42ec-4f0e-92a3-8d95fd31154b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:01.850705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295fbef-42ec-4f0e-92a3-8d95fd31154b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:01.857240 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:01.869762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8295fbef-42ec-4f0e-92a3-8d95fd31154b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:11:01.870974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8295fbef-42ec-4f0e-92a3-8d95fd31154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:11:01.967118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 616/1248] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:11:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:11:02.240402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35381624-71fc-4421-9c57-b571f57740e6 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:02.261354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:02.364452 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35381624-71fc-4421-9c57-b571f57740e6 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:02.404410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4bf29-16ea-4ac1-936c-fe2b7c4294f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:02.405033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4bf29-16ea-4ac1-936c-fe2b7c4294f6 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:02.405311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4bf29-16ea-4ac1-936c-fe2b7c4294f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:02.405584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4bf29-16ea-4ac1-936c-fe2b7c4294f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:02.416828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4bf29-16ea-4ac1-936c-fe2b7c4294f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-dae4bf29-16ea-4ac1-936c-fe2b7c4294f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:11:02.422228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:02.423491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:11:02.424449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 633/1249] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:11:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:02.429645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295fbef-42ec-4f0e-92a3-8d95fd31154b None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:02.532584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295fbef-42ec-4f0e-92a3-8d95fd31154b None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:02.563869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:02.568928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557a575-7211-4533-b864-fcadc27d0315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:02.569545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557a575-7211-4533-b864-fcadc27d0315 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:02.569826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557a575-7211-4533-b864-fcadc27d0315 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:02.570156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557a575-7211-4533-b864-fcadc27d0315 None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:02.570777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557a575-7211-4533-b864-fcadc27d0315 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:02.571082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557a575-7211-4533-b864-fcadc27d0315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:02.572503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557a575-7211-4533-b864-fcadc27d0315 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:02.581626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 617/1250] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:11:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:02.587722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4bf29-16ea-4ac1-936c-fe2b7c4294f6 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:02.671224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:02.694768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4bf29-16ea-4ac1-936c-fe2b7c4294f6 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:02.730430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983a35e-9fd5-4c76-b528-54730d3892af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:02.731076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983a35e-9fd5-4c76-b528-54730d3892af None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:02.731362 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983a35e-9fd5-4c76-b528-54730d3892af None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:02.731668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983a35e-9fd5-4c76-b528-54730d3892af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:02.743016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983a35e-9fd5-4c76-b528-54730d3892af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:02.744625 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983a35e-9fd5-4c76-b528-54730d3892af None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-f983a35e-9fd5-4c76-b528-54730d3892af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:11:02.759495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:02.760737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:11:02.761709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:11:02.761709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:02.761709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:11:02.761709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 634/1251] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:02.767004 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557a575-7211-4533-b864-fcadc27d0315 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:02.874400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557a575-7211-4533-b864-fcadc27d0315 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:02.912149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3e529-4c78-4797-a194-77cd21394033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:02.912769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3e529-4c78-4797-a194-77cd21394033 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:02.913055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3e529-4c78-4797-a194-77cd21394033 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:02.913332 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3e529-4c78-4797-a194-77cd21394033 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:02.914063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3e529-4c78-4797-a194-77cd21394033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:02.914368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3e529-4c78-4797-a194-77cd21394033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:02.915815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3e529-4c78-4797-a194-77cd21394033 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:02.928732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 618/1252] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:02.934791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983a35e-9fd5-4c76-b528-54730d3892af None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:03.040657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983a35e-9fd5-4c76-b528-54730d3892af None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:03.069411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:03.078252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b7d8b-8bb7-45ad-b444-0ef2a1cce047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:03.078956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b7d8b-8bb7-45ad-b444-0ef2a1cce047 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:03.079184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b7d8b-8bb7-45ad-b444-0ef2a1cce047 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:03.079458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b7d8b-8bb7-45ad-b444-0ef2a1cce047 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:03.087543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b7d8b-8bb7-45ad-b444-0ef2a1cce047 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:03.089026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b7d8b-8bb7-45ad-b444-0ef2a1cce047 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-5a0b7d8b-8bb7-45ad-b444-0ef2a1cce047 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:11:03.101537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:03.103197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:11:03.104275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 635/1253] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:11:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:03.110360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3e529-4c78-4797-a194-77cd21394033 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:03.172455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:03.219696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3e529-4c78-4797-a194-77cd21394033 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:03.256865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39863a-2fb9-400d-8ce5-1cc0bccfe400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:03.257590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39863a-2fb9-400d-8ce5-1cc0bccfe400 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:03.257873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39863a-2fb9-400d-8ce5-1cc0bccfe400 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:03.258151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39863a-2fb9-400d-8ce5-1cc0bccfe400 None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:03.258896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39863a-2fb9-400d-8ce5-1cc0bccfe400 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:03.259210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39863a-2fb9-400d-8ce5-1cc0bccfe400 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:03.260724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39863a-2fb9-400d-8ce5-1cc0bccfe400 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:03.279718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 619/1254] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Sep 23 01:11:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:03.285688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b7d8b-8bb7-45ad-b444-0ef2a1cce047 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:03.393110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b7d8b-8bb7-45ad-b444-0ef2a1cce047 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:03.437664 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b177760-e512-4159-a0b5-bba5a6fe70bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:03.438376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b177760-e512-4159-a0b5-bba5a6fe70bb None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:03.438626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b177760-e512-4159-a0b5-bba5a6fe70bb None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:03.438907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b177760-e512-4159-a0b5-bba5a6fe70bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:03.513472 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:03.527788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b177760-e512-4159-a0b5-bba5a6fe70bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:03.529297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b177760-e512-4159-a0b5-bba5a6fe70bb None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:03.635341 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 636/1255] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:11:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:11:03.642537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39863a-2fb9-400d-8ce5-1cc0bccfe400 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:03.755797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39863a-2fb9-400d-8ce5-1cc0bccfe400 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:03.775345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:03.799405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5fbbf-4640-4ca2-b4c7-209296e6e015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:03.800081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5fbbf-4640-4ca2-b4c7-209296e6e015 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:03.800359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5fbbf-4640-4ca2-b4c7-209296e6e015 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:03.800630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5fbbf-4640-4ca2-b4c7-209296e6e015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:03.813187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5fbbf-4640-4ca2-b4c7-209296e6e015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:03.814871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5fbbf-4640-4ca2-b4c7-209296e6e015 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-cde5fbbf-4640-4ca2-b4c7-209296e6e015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:11:03.831787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:03.834548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:11:03.835759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:11:03.835759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:03.835759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:03.835759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 620/1256] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:03.841738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b177760-e512-4159-a0b5-bba5a6fe70bb None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:03.940275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:03.957185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b177760-e512-4159-a0b5-bba5a6fe70bb None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:03.994453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64199f-38be-41bf-ba88-0dd792f9768d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:03.995063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64199f-38be-41bf-ba88-0dd792f9768d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:03.995340 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64199f-38be-41bf-ba88-0dd792f9768d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:03.995636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64199f-38be-41bf-ba88-0dd792f9768d None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:03.996278 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64199f-38be-41bf-ba88-0dd792f9768d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:03.996575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64199f-38be-41bf-ba88-0dd792f9768d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:03.997946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a64199f-38be-41bf-ba88-0dd792f9768d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:04.009631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 637/1257] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:04.015388 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5fbbf-4640-4ca2-b4c7-209296e6e015 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:04.121535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5fbbf-4640-4ca2-b4c7-209296e6e015 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:04.155502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db562946-900a-40f2-b0cd-40ce39725005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:04.159713 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db562946-900a-40f2-b0cd-40ce39725005 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:04.159713 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db562946-900a-40f2-b0cd-40ce39725005 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:04.159713 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db562946-900a-40f2-b0cd-40ce39725005 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:04.164924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db562946-900a-40f2-b0cd-40ce39725005 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:04.166370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db562946-900a-40f2-b0cd-40ce39725005 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-db562946-900a-40f2-b0cd-40ce39725005 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:11:04.178525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:04.179681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:04.180888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:11:04.180888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:11:04.180888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:11:04.180888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:11:04.180888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:11:04.180888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:11:04.180888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:11:04.180888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:04.180888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 621/1258] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:11:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:04.185826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64199f-38be-41bf-ba88-0dd792f9768d None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:04.290236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64199f-38be-41bf-ba88-0dd792f9768d None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:04.325034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7fcf1-7745-4185-862b-cd607c0a058f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:04.325671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fcf1-7745-4185-862b-cd607c0a058f None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:04.325955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fcf1-7745-4185-862b-cd607c0a058f None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:04.326231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fcf1-7745-4185-862b-cd607c0a058f None admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:04.326932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7fcf1-7745-4185-862b-cd607c0a058f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:04.327240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7fcf1-7745-4185-862b-cd607c0a058f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:04.329230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7fcf1-7745-4185-862b-cd607c0a058f None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:04.341412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 638/1259] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:11:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:04.348465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db562946-900a-40f2-b0cd-40ce39725005 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:04.375746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:04.449888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db562946-900a-40f2-b0cd-40ce39725005 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:04.485472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e3927-29be-4dc2-91c6-d50c50dbc69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:04.486075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3927-29be-4dc2-91c6-d50c50dbc69c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:04.486349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3927-29be-4dc2-91c6-d50c50dbc69c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:04.486665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e3927-29be-4dc2-91c6-d50c50dbc69c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:04.487394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e3927-29be-4dc2-91c6-d50c50dbc69c None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=9f73bca3896440258430cfce042faa42) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:04.487790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e3927-29be-4dc2-91c6-d50c50dbc69c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:04.489218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e3927-29be-4dc2-91c6-d50c50dbc69c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:04.494835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 622/1260] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:11:04 2022] GET /identity/v3/role_assignments?user.id=9f73bca3896440258430cfce042faa42&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:06.677072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e8a7-0b59-48f1-aa58-cb0c0c929aaf None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:06.677718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e8a7-0b59-48f1-aa58-cb0c0c929aaf None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:06.677718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8e8a7-0b59-48f1-aa58-cb0c0c929aaf None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:06.679056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 639/1261] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:11:06.684739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943e3c3-66ea-462c-8046-f81353bc01be None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:06.685293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943e3c3-66ea-462c-8046-f81353bc01be None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:06.685390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a943e3c3-66ea-462c-8046-f81353bc01be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:06.705522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a943e3c3-66ea-462c-8046-f81353bc01be None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:11:06.710033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a943e3c3-66ea-462c-8046-f81353bc01be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:11:06.781036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 623/1262] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:11:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:11:07.057460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8e8a7-0b59-48f1-aa58-cb0c0c929aaf None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:07.069268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:07.200172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8e8a7-0b59-48f1-aa58-cb0c0c929aaf None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:07.245199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06601bd4-1f07-4d40-8cba-1768112fe61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:07.245836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06601bd4-1f07-4d40-8cba-1768112fe61e None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:07.246115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06601bd4-1f07-4d40-8cba-1768112fe61e None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:07.246388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06601bd4-1f07-4d40-8cba-1768112fe61e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:07.257180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06601bd4-1f07-4d40-8cba-1768112fe61e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-06601bd4-1f07-4d40-8cba-1768112fe61e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:11:07.262065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:07.263345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:11:07.264597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 640/1263] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:11:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:07.269643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943e3c3-66ea-462c-8046-f81353bc01be None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:07.288508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:07.377503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a943e3c3-66ea-462c-8046-f81353bc01be None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:07.414497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f55b7d-f146-4941-a610-bb3ada27d2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:07.415105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f55b7d-f146-4941-a610-bb3ada27d2df None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:07.415381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f55b7d-f146-4941-a610-bb3ada27d2df None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:07.415680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f55b7d-f146-4941-a610-bb3ada27d2df None admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:07.416355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f55b7d-f146-4941-a610-bb3ada27d2df None admin] RBAC: Adding query filter params (name=admin) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:07.416651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f55b7d-f146-4941-a610-bb3ada27d2df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:07.418024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f55b7d-f146-4941-a610-bb3ada27d2df None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:07.426861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 624/1264] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:11:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:07.433102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06601bd4-1f07-4d40-8cba-1768112fe61e None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:07.520151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:07.554042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06601bd4-1f07-4d40-8cba-1768112fe61e None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:07.589500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb64ec-1432-40f7-843c-a3807be2d411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:07.590144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb64ec-1432-40f7-843c-a3807be2d411 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:07.590427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb64ec-1432-40f7-843c-a3807be2d411 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:07.590696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb64ec-1432-40f7-843c-a3807be2d411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:07.601688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb64ec-1432-40f7-843c-a3807be2d411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:07.603141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb64ec-1432-40f7-843c-a3807be2d411 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-dfcb64ec-1432-40f7-843c-a3807be2d411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:11:07.618317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:07.619498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:11:07.620710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:11:07.620710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:07.620710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:11:07.620710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 641/1265] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:07.625587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f55b7d-f146-4941-a610-bb3ada27d2df None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:07.714169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:07.734425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f55b7d-f146-4941-a610-bb3ada27d2df None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:07.771046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d32108-cfc3-47a7-8969-78cb0481c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:07.771680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d32108-cfc3-47a7-8969-78cb0481c283 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:07.771974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d32108-cfc3-47a7-8969-78cb0481c283 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:07.772236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d32108-cfc3-47a7-8969-78cb0481c283 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:07.772884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d32108-cfc3-47a7-8969-78cb0481c283 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:07.773183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d32108-cfc3-47a7-8969-78cb0481c283 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:07.774568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d32108-cfc3-47a7-8969-78cb0481c283 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:07.786454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 625/1266] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:07.792411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb64ec-1432-40f7-843c-a3807be2d411 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:07.899271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb64ec-1432-40f7-843c-a3807be2d411 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:07.920315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:07.934504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce5acd-cf6f-4eec-848d-cb10319072d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:07.935157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce5acd-cf6f-4eec-848d-cb10319072d5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:07.935438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce5acd-cf6f-4eec-848d-cb10319072d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:07.935755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce5acd-cf6f-4eec-848d-cb10319072d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:07.950603 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce5acd-cf6f-4eec-848d-cb10319072d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:07.953958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce5acd-cf6f-4eec-848d-cb10319072d5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-02ce5acd-cf6f-4eec-848d-cb10319072d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:11:07.971904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:07.973149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:07.974435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:07.974435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:07.974435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:11:07.974435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:11:07.974435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:11:07.974435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:11:07.974435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:11:07.974435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:11:07.974435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:11:07.974435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:11:07.975739 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 642/1267] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:11:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:07.978789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d32108-cfc3-47a7-8969-78cb0481c283 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:08.087258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d32108-cfc3-47a7-8969-78cb0481c283 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:08.126050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e387-8042-426a-b0f5-60174d71b83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:08.126659 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e387-8042-426a-b0f5-60174d71b83f None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:08.126939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e387-8042-426a-b0f5-60174d71b83f None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:08.127217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058e387-8042-426a-b0f5-60174d71b83f None admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:08.127987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058e387-8042-426a-b0f5-60174d71b83f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:08.128298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058e387-8042-426a-b0f5-60174d71b83f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:08.129743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058e387-8042-426a-b0f5-60174d71b83f None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:08.139291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:08.148157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 626/1268] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Sep 23 01:11:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:08.153977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce5acd-cf6f-4eec-848d-cb10319072d5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:08.267043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce5acd-cf6f-4eec-848d-cb10319072d5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:08.303345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97113527-ce58-488e-94c5-dc1afd67952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:08.304038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97113527-ce58-488e-94c5-dc1afd67952b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:08.304315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97113527-ce58-488e-94c5-dc1afd67952b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:08.304590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97113527-ce58-488e-94c5-dc1afd67952b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:08.316990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97113527-ce58-488e-94c5-dc1afd67952b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:08.318442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97113527-ce58-488e-94c5-dc1afd67952b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-97113527-ce58-488e-94c5-dc1afd67952b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:11:08.334263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:08.335583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:11:08.336530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:11:08.336530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:08.336530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:11:08.336530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 643/1269] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:08.341444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e387-8042-426a-b0f5-60174d71b83f None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:08.447071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058e387-8042-426a-b0f5-60174d71b83f None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:08.488464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88a2e9-f8b4-4605-b8a3-f036cdcb7698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:08.489112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88a2e9-f8b4-4605-b8a3-f036cdcb7698 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:08.489391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88a2e9-f8b4-4605-b8a3-f036cdcb7698 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:08.489686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f88a2e9-f8b4-4605-b8a3-f036cdcb7698 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:08.490347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88a2e9-f8b4-4605-b8a3-f036cdcb7698 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:08.490655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88a2e9-f8b4-4605-b8a3-f036cdcb7698 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:08.492087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f88a2e9-f8b4-4605-b8a3-f036cdcb7698 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:08.504303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 627/1270] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:08.510158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97113527-ce58-488e-94c5-dc1afd67952b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:08.548061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:08.618692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97113527-ce58-488e-94c5-dc1afd67952b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:08.656523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2278e-7b58-41d1-829b-5c68f77c6d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:08.657184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2278e-7b58-41d1-829b-5c68f77c6d0b None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:08.657457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2278e-7b58-41d1-829b-5c68f77c6d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:08.657730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2278e-7b58-41d1-829b-5c68f77c6d0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:08.665577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2278e-7b58-41d1-829b-5c68f77c6d0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:08.667058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2278e-7b58-41d1-829b-5c68f77c6d0b None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-a9b2278e-7b58-41d1-829b-5c68f77c6d0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:11:08.680611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:08.682462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:08.683650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:11:08.683650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:11:08.683650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:11:08.683650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:11:08.683650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:11:08.683650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:11:08.683650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:11:08.683650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:11:08.683650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 644/1271] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:08.688113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88a2e9-f8b4-4605-b8a3-f036cdcb7698 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:08.753238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:08.801107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f88a2e9-f8b4-4605-b8a3-f036cdcb7698 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:08.836683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09e663-279a-4ad0-82ca-d9a7fe58a0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:08.837295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e663-279a-4ad0-82ca-d9a7fe58a0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:08.837574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e663-279a-4ad0-82ca-d9a7fe58a0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:08.837849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09e663-279a-4ad0-82ca-d9a7fe58a0b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:08.838548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09e663-279a-4ad0-82ca-d9a7fe58a0b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:08.838855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09e663-279a-4ad0-82ca-d9a7fe58a0b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:08.840346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09e663-279a-4ad0-82ca-d9a7fe58a0b4 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:08.853599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 628/1272] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:08.859883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2278e-7b58-41d1-829b-5c68f77c6d0b None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:08.960764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:08.972104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2278e-7b58-41d1-829b-5c68f77c6d0b None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:09.007170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63193dfa-5ca0-4ec1-b1cb-203b630077ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:09.008038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63193dfa-5ca0-4ec1-b1cb-203b630077ac None admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:09.008315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63193dfa-5ca0-4ec1-b1cb-203b630077ac None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:09.008589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63193dfa-5ca0-4ec1-b1cb-203b630077ac None admin] PATH_INFO: `/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/9f73bca3896440258430cfce042faa42/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:09.010634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63193dfa-5ca0-4ec1-b1cb-203b630077ac None admin] RBAC: Authorizing `identity:create_grant(project_id=29f2f05139514a648667c0846ff3d7b7, user_id=9f73bca3896440258430cfce042faa42, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:09.012110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63193dfa-5ca0-4ec1-b1cb-203b630077ac None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:09.138683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 645/1273] 10.222.0.64 () {58 vars in 1439 bytes} [Fri Sep 23 01:11:08 2022] PUT /identity/v3/projects/29f2f05139514a648667c0846ff3d7b7/users/9f73bca3896440258430cfce042faa42/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:11:11.301096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85254e4-f547-4bf6-87fb-c4b4e4da639a None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:11.301653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85254e4-f547-4bf6-87fb-c4b4e4da639a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:11.301653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85254e4-f547-4bf6-87fb-c4b4e4da639a None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:11.303106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 629/1274] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:11:11.308790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913c201-8f45-428d-935f-6e42fb2d2358 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:11.309268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913c201-8f45-428d-935f-6e42fb2d2358 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:11.309550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913c201-8f45-428d-935f-6e42fb2d2358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:11.330145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f913c201-8f45-428d-935f-6e42fb2d2358 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:11:11.331358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f913c201-8f45-428d-935f-6e42fb2d2358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:11:11.400796 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 646/1275] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:11:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:11:11.679418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85254e4-f547-4bf6-87fb-c4b4e4da639a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:11.808280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85254e4-f547-4bf6-87fb-c4b4e4da639a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:11.834335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:11.853360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bd38f-e0fa-4aaf-9259-1e4227b2ed7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:11.854068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bd38f-e0fa-4aaf-9259-1e4227b2ed7c None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:11.854300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bd38f-e0fa-4aaf-9259-1e4227b2ed7c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:11.854574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bd38f-e0fa-4aaf-9259-1e4227b2ed7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:11.873865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1bd38f-e0fa-4aaf-9259-1e4227b2ed7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-bd1bd38f-e0fa-4aaf-9259-1e4227b2ed7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 23 01:11:11.880544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:11.881876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:11.882757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 630/1276] 10.222.0.64 () {56 vars in 1096 bytes} [Fri Sep 23 01:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:11.888192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913c201-8f45-428d-935f-6e42fb2d2358 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:11.976634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:11.994522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913c201-8f45-428d-935f-6e42fb2d2358 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:12.032478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedf31d-efe4-48b1-a50f-1371ae59b2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:12.033104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedf31d-efe4-48b1-a50f-1371ae59b2fc None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:12.033400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedf31d-efe4-48b1-a50f-1371ae59b2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:12.033686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cedf31d-efe4-48b1-a50f-1371ae59b2fc None admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:12.034327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedf31d-efe4-48b1-a50f-1371ae59b2fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:12.034631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedf31d-efe4-48b1-a50f-1371ae59b2fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:12.036130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cedf31d-efe4-48b1-a50f-1371ae59b2fc None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:12.044391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 647/1277] 10.222.0.64 () {56 vars in 1099 bytes} [Fri Sep 23 01:11:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:12.050609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bd38f-e0fa-4aaf-9259-1e4227b2ed7c None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:12.151663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bd38f-e0fa-4aaf-9259-1e4227b2ed7c None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:12.185947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21c3e1-9035-4124-b314-291bf3600879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:12.186564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21c3e1-9035-4124-b314-291bf3600879 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:12.186862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21c3e1-9035-4124-b314-291bf3600879 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:12.187137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21c3e1-9035-4124-b314-291bf3600879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:12.197551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21c3e1-9035-4124-b314-291bf3600879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:12.199017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21c3e1-9035-4124-b314-291bf3600879 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-df21c3e1-9035-4124-b314-291bf3600879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:11:12.214575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:12.215821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:11:12.216912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:11:12.216912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:12.216912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:12.216912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 631/1278] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:11:12.222479 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedf31d-efe4-48b1-a50f-1371ae59b2fc None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:12.323939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cedf31d-efe4-48b1-a50f-1371ae59b2fc None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:12.360040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b1d2d-5742-4ca0-8786-effa835567e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:12.360649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b1d2d-5742-4ca0-8786-effa835567e4 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:12.360929 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b1d2d-5742-4ca0-8786-effa835567e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:12.361202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b1d2d-5742-4ca0-8786-effa835567e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:12.361845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b1d2d-5742-4ca0-8786-effa835567e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:12.362151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b1d2d-5742-4ca0-8786-effa835567e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:12.363517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b1d2d-5742-4ca0-8786-effa835567e4 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:12.376682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 648/1279] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:12.382640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21c3e1-9035-4124-b314-291bf3600879 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:12.414346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:12.486447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21c3e1-9035-4124-b314-291bf3600879 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:12.521850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92825226-133d-4735-b2d4-a3e3491f42e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:12.522478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92825226-133d-4735-b2d4-a3e3491f42e8 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:12.522760 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92825226-133d-4735-b2d4-a3e3491f42e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:12.523032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92825226-133d-4735-b2d4-a3e3491f42e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:12.530526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92825226-133d-4735-b2d4-a3e3491f42e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:12.532015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92825226-133d-4735-b2d4-a3e3491f42e8 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-92825226-133d-4735-b2d4-a3e3491f42e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 23 01:11:12.544287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:12.545560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:12.546618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 632/1280] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:11:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:12.551680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b1d2d-5742-4ca0-8786-effa835567e4 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:12.555826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:12.653764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b1d2d-5742-4ca0-8786-effa835567e4 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:12.688021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865d74d-d5eb-42c3-bfc3-d6245609fdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:12.688626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865d74d-d5eb-42c3-bfc3-d6245609fdb5 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:12.688899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865d74d-d5eb-42c3-bfc3-d6245609fdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:12.689167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e865d74d-d5eb-42c3-bfc3-d6245609fdb5 None admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:12.689891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865d74d-d5eb-42c3-bfc3-d6245609fdb5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:12.690193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865d74d-d5eb-42c3-bfc3-d6245609fdb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:12.691633 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e865d74d-d5eb-42c3-bfc3-d6245609fdb5 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:12.709302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 649/1281] 10.222.0.64 () {56 vars in 1143 bytes} [Fri Sep 23 01:11:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:12.715599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92825226-133d-4735-b2d4-a3e3491f42e8 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:12.816194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:12.831726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92825226-133d-4735-b2d4-a3e3491f42e8 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:12.866376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6e0e1-ff94-4d4f-acc8-0af98dea9def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:12.866985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6e0e1-ff94-4d4f-acc8-0af98dea9def None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:12.867265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6e0e1-ff94-4d4f-acc8-0af98dea9def None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:12.867538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6e0e1-ff94-4d4f-acc8-0af98dea9def None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:12.939583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6e0e1-ff94-4d4f-acc8-0af98dea9def None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:12.940987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6e0e1-ff94-4d4f-acc8-0af98dea9def None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:13.051740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:13.065420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 633/1282] 10.222.0.64 () {58 vars in 1282 bytes} [Fri Sep 23 01:11:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 23 01:11:13.071911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865d74d-d5eb-42c3-bfc3-d6245609fdb5 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:13.134872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:13.173821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e865d74d-d5eb-42c3-bfc3-d6245609fdb5 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:13.207888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26064f-cea9-4e2a-b27b-7e547ca85b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:13.208434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26064f-cea9-4e2a-b27b-7e547ca85b22 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:13.208711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26064f-cea9-4e2a-b27b-7e547ca85b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:13.208983 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26064f-cea9-4e2a-b27b-7e547ca85b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:13.221349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26064f-cea9-4e2a-b27b-7e547ca85b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:13.223004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f26064f-cea9-4e2a-b27b-7e547ca85b22 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-7f26064f-cea9-4e2a-b27b-7e547ca85b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 23 01:11:13.239911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:13.241287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 23 01:11:13.242408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 23 01:11:13.242408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 23 01:11:13.242408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:11:13.242408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 650/1283] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:13.247342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6e0e1-ff94-4d4f-acc8-0af98dea9def None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:13.352082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6e0e1-ff94-4d4f-acc8-0af98dea9def None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:13.388213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dfbad-77b8-42bd-8b45-2042b81476f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:13.388930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dfbad-77b8-42bd-8b45-2042b81476f1 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:13.389199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dfbad-77b8-42bd-8b45-2042b81476f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:13.389481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dfbad-77b8-42bd-8b45-2042b81476f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:13.390674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502dfbad-77b8-42bd-8b45-2042b81476f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:13.390979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502dfbad-77b8-42bd-8b45-2042b81476f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:13.392407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502dfbad-77b8-42bd-8b45-2042b81476f1 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:13.404490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 634/1284] 10.222.0.64 () {56 vars in 1109 bytes} [Fri Sep 23 01:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:13.411093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26064f-cea9-4e2a-b27b-7e547ca85b22 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:13.518529 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f26064f-cea9-4e2a-b27b-7e547ca85b22 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:13.533725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:13.553513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01179b38-fa6a-46ec-a64b-eae53c7b0ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:13.554138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01179b38-fa6a-46ec-a64b-eae53c7b0ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:13.554417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01179b38-fa6a-46ec-a64b-eae53c7b0ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:13.554689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01179b38-fa6a-46ec-a64b-eae53c7b0ea4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:13.561243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01179b38-fa6a-46ec-a64b-eae53c7b0ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:13.562752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01179b38-fa6a-46ec-a64b-eae53c7b0ea4 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application [None req-01179b38-fa6a-46ec-a64b-eae53c7b0ea4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 23 01:11:13.575177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:13.576396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:13.578053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 23 01:11:13.578053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 23 01:11:13.578053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 23 01:11:13.578053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 23 01:11:13.578053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 23 01:11:13.578053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 23 01:11:13.578053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 23 01:11:13.578053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: ERROR keystone.server.flask.application Sep 23 01:11:13.578053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 651/1285] 10.222.0.64 () {56 vars in 1111 bytes} [Fri Sep 23 01:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:13.582961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dfbad-77b8-42bd-8b45-2042b81476f1 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:13.633244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:13.682408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dfbad-77b8-42bd-8b45-2042b81476f1 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:13.716113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3f1c7-6049-4dfd-a0dd-24befcf94bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:13.716718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3f1c7-6049-4dfd-a0dd-24befcf94bfc None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:13.716997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3f1c7-6049-4dfd-a0dd-24befcf94bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:13.717270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3f1c7-6049-4dfd-a0dd-24befcf94bfc None admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:13.717963 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3f1c7-6049-4dfd-a0dd-24befcf94bfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:13.718904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3f1c7-6049-4dfd-a0dd-24befcf94bfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:13.720358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3f1c7-6049-4dfd-a0dd-24befcf94bfc None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:13.732790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 635/1286] 10.222.0.64 () {56 vars in 1148 bytes} [Fri Sep 23 01:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:13.738753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01179b38-fa6a-46ec-a64b-eae53c7b0ea4 None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:13.843709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01179b38-fa6a-46ec-a64b-eae53c7b0ea4 None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:13.881247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d8a5c-0b9f-403b-b474-123443f4ce4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:13.881888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d8a5c-0b9f-403b-b474-123443f4ce4d None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:13.882169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d8a5c-0b9f-403b-b474-123443f4ce4d None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:13.882444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d8a5c-0b9f-403b-b474-123443f4ce4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:13.883189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d8a5c-0b9f-403b-b474-123443f4ce4d None admin] RBAC: Adding query filter params (role.id=8928ed5fc98f44048f4c1b90fae793b6, scope.project.id=29f2f05139514a648667c0846ff3d7b7, user.id=9f73bca3896440258430cfce042faa42) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:13.883502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d8a5c-0b9f-403b-b474-123443f4ce4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:13.884968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d8a5c-0b9f-403b-b474-123443f4ce4d None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:13.891852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 652/1287] 10.222.0.64 () {56 vars in 1374 bytes} [Fri Sep 23 01:11:13 2022] GET /identity/v3/role_assignments?user.id=9f73bca3896440258430cfce042faa42&scope.project.id=29f2f05139514a648667c0846ff3d7b7&role.id=8928ed5fc98f44048f4c1b90fae793b6 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:16.132935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2b8dd-af3e-4464-aece-067862a02486 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:16.133572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2b8dd-af3e-4464-aece-067862a02486 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:16.133572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2b8dd-af3e-4464-aece-067862a02486 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:16.134859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 636/1288] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:11:16.140998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e41625-a14c-4e36-943c-665f86379746 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:16.141425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e41625-a14c-4e36-943c-665f86379746 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:16.141542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e41625-a14c-4e36-943c-665f86379746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:16.160390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-49e41625-a14c-4e36-943c-665f86379746 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:11:16.161590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-49e41625-a14c-4e36-943c-665f86379746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:11:16.192004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:16.230558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 653/1289] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:11:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:11:16.504274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2b8dd-af3e-4464-aece-067862a02486 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:16.633150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2b8dd-af3e-4464-aece-067862a02486 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:16.645685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:16.669555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8889bb3-7790-424d-b085-01e924e03f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:16.670181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8889bb3-7790-424d-b085-01e924e03f26 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:16.670457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8889bb3-7790-424d-b085-01e924e03f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:16.670731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8889bb3-7790-424d-b085-01e924e03f26 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:16.671856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8889bb3-7790-424d-b085-01e924e03f26 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:16.673244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8889bb3-7790-424d-b085-01e924e03f26 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-c8889bb3-7790-424d-b085-01e924e03f26 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:16.682349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:16.683718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:16.696253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 637/1290] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Sep 23 01:11:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:16.697273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e41625-a14c-4e36-943c-665f86379746 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:16.794135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e41625-a14c-4e36-943c-665f86379746 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:16.829384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e43750-f194-4b91-bbbd-5e242d3c9ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:16.830273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e43750-f194-4b91-bbbd-5e242d3c9ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:16.830372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e43750-f194-4b91-bbbd-5e242d3c9ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:16.830592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e43750-f194-4b91-bbbd-5e242d3c9ef1 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:16.831239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e43750-f194-4b91-bbbd-5e242d3c9ef1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:16.831540 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e43750-f194-4b91-bbbd-5e242d3c9ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:16.832953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e43750-f194-4b91-bbbd-5e242d3c9ef1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:16.838305 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 654/1291] 10.222.0.64 () {56 vars in 1116 bytes} [Fri Sep 23 01:11:16 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:16.844763 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8889bb3-7790-424d-b085-01e924e03f26 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:16.950908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8889bb3-7790-424d-b085-01e924e03f26 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:16.992929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6bac1-b70a-4419-8a5d-e62d71a08862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:16.993526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6bac1-b70a-4419-8a5d-e62d71a08862 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:16.993807 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6bac1-b70a-4419-8a5d-e62d71a08862 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:16.994081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6bac1-b70a-4419-8a5d-e62d71a08862 None admin] PATH_INFO: `/v3/services` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:16.994719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6bac1-b70a-4419-8a5d-e62d71a08862 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:16.995023 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6bac1-b70a-4419-8a5d-e62d71a08862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:16.996530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6bac1-b70a-4419-8a5d-e62d71a08862 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:17.001643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 638/1292] 10.222.0.64 () {56 vars in 1116 bytes} [Fri Sep 23 01:11:16 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:19.219215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0da51-b1ec-4fc4-a8b2-04453260741f None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:19.219775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0da51-b1ec-4fc4-a8b2-04453260741f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:19.219913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0da51-b1ec-4fc4-a8b2-04453260741f None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:19.221185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 655/1293] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:11:19.226760 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a6c1-f046-4617-8fa1-f028e10fd799 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:19.228544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a6c1-f046-4617-8fa1-f028e10fd799 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:19.228544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1a6c1-f046-4617-8fa1-f028e10fd799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:19.246521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-28e1a6c1-f046-4617-8fa1-f028e10fd799 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:11:19.247772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-28e1a6c1-f046-4617-8fa1-f028e10fd799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:11:19.279432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:19.322539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 639/1294] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:11:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:11:19.599512 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0da51-b1ec-4fc4-a8b2-04453260741f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:19.633649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:19.710707 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0da51-b1ec-4fc4-a8b2-04453260741f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:19.751156 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd90238-c039-41b6-ba98-0a8f1aded8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:19.751821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd90238-c039-41b6-ba98-0a8f1aded8c1 None admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:19.752103 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd90238-c039-41b6-ba98-0a8f1aded8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:19.752382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd90238-c039-41b6-ba98-0a8f1aded8c1 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:19.753136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd90238-c039-41b6-ba98-0a8f1aded8c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:19.754521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd90238-c039-41b6-ba98-0a8f1aded8c1 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:19.919233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 656/1295] 10.222.0.64 () {60 vars in 1141 bytes} [Fri Sep 23 01:11:19 2022] POST /identity/v3/services => generated 246 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:11:22.203017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9615c26-bffd-42ce-99a8-dfc539df7505 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:22.203591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9615c26-bffd-42ce-99a8-dfc539df7505 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:22.203591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9615c26-bffd-42ce-99a8-dfc539df7505 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:22.204976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 640/1296] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:11:22.210567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f8767-574a-4ff7-97d6-76eab07475f6 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:22.211054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f8767-574a-4ff7-97d6-76eab07475f6 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:22.211149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f8767-574a-4ff7-97d6-76eab07475f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:22.230579 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-837f8767-574a-4ff7-97d6-76eab07475f6 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:11:22.231873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-837f8767-574a-4ff7-97d6-76eab07475f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:11:22.308710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 657/1297] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:11:22 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:11:22.584450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9615c26-bffd-42ce-99a8-dfc539df7505 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:22.704079 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9615c26-bffd-42ce-99a8-dfc539df7505 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:22.742342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407292a-6ca1-4449-b56b-fd0cb9cb5b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:22.742988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407292a-6ca1-4449-b56b-fd0cb9cb5b21 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:22.743271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407292a-6ca1-4449-b56b-fd0cb9cb5b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:22.743542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1407292a-6ca1-4449-b56b-fd0cb9cb5b21 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:22.744162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407292a-6ca1-4449-b56b-fd0cb9cb5b21 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:22.745554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1407292a-6ca1-4449-b56b-fd0cb9cb5b21 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-1407292a-6ca1-4449-b56b-fd0cb9cb5b21 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:22.755602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:22.756897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:22.758026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 641/1298] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Sep 23 01:11:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 23 01:11:22.763220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f8767-574a-4ff7-97d6-76eab07475f6 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:22.783559 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:22.869155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f8767-574a-4ff7-97d6-76eab07475f6 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:22.907919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88eb1f-3bc6-47ba-b727-2fa3df4471ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:22.908537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88eb1f-3bc6-47ba-b727-2fa3df4471ca None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:22.908813 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88eb1f-3bc6-47ba-b727-2fa3df4471ca None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:22.909104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88eb1f-3bc6-47ba-b727-2fa3df4471ca None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:22.909718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88eb1f-3bc6-47ba-b727-2fa3df4471ca None admin] RBAC: Adding query filter params (name=placement) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:22.910017 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88eb1f-3bc6-47ba-b727-2fa3df4471ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:22.911386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88eb1f-3bc6-47ba-b727-2fa3df4471ca None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:22.918137 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 658/1299] 10.222.0.64 () {56 vars in 1116 bytes} [Fri Sep 23 01:11:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:11:22.924256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407292a-6ca1-4449-b56b-fd0cb9cb5b21 None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:22.958251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:23.039750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1407292a-6ca1-4449-b56b-fd0cb9cb5b21 None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:23.076040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1455ff0e-5da1-4dba-9e1f-add356db4650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:23.076656 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455ff0e-5da1-4dba-9e1f-add356db4650 None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:23.077010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455ff0e-5da1-4dba-9e1f-add356db4650 None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:23.077285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1455ff0e-5da1-4dba-9e1f-add356db4650 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:23.078640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455ff0e-5da1-4dba-9e1f-add356db4650 None admin] RBAC: Adding query filter params (interface=public, service_id=10dca8d3646940df8bf955a430b97f3f, region_id=RegionOne) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:23.078949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455ff0e-5da1-4dba-9e1f-add356db4650 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:23.080373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1455ff0e-5da1-4dba-9e1f-add356db4650 None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:23.086111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 642/1300] 10.222.0.64 () {56 vars in 1251 bytes} [Fri Sep 23 01:11:22 2022] GET /identity/v3/endpoints?service_id=10dca8d3646940df8bf955a430b97f3f&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:23.092573 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88eb1f-3bc6-47ba-b727-2fa3df4471ca None admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:23.191679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:23.211906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88eb1f-3bc6-47ba-b727-2fa3df4471ca None admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:23.250216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ff6a7-9c86-4737-9923-a45c2dadbb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:23.250904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ff6a7-9c86-4737-9923-a45c2dadbb50 None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:23.251232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ff6a7-9c86-4737-9923-a45c2dadbb50 None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:23.251593 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ff6a7-9c86-4737-9923-a45c2dadbb50 None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:23.252318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ff6a7-9c86-4737-9923-a45c2dadbb50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:23.254260 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ff6a7-9c86-4737-9923-a45c2dadbb50 None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:23.262742 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 659/1301] 10.222.0.64 () {56 vars in 1087 bytes} [Fri Sep 23 01:11:23 2022] GET /identity/v3/services => generated 2191 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:11:25.467470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd43227-97d8-44cd-95f6-984aabaa828e None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:25.468984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd43227-97d8-44cd-95f6-984aabaa828e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:25.468984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd43227-97d8-44cd-95f6-984aabaa828e None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:25.470642 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 643/1302] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:11:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:11:25.476323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477331b-eedf-486a-8917-4e321f6d7187 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:25.476799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477331b-eedf-486a-8917-4e321f6d7187 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:25.476891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477331b-eedf-486a-8917-4e321f6d7187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:25.495713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c477331b-eedf-486a-8917-4e321f6d7187 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:11:25.496927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c477331b-eedf-486a-8917-4e321f6d7187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:11:25.571689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 660/1303] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:11:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 23 01:11:25.845894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd43227-97d8-44cd-95f6-984aabaa828e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:25.948465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:25.956307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd43227-97d8-44cd-95f6-984aabaa828e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:25.989885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0861d31-b27b-4807-87e4-dad70ba56d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:25.990513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0861d31-b27b-4807-87e4-dad70ba56d4b None admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:25.990794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0861d31-b27b-4807-87e4-dad70ba56d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:25.991069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0861d31-b27b-4807-87e4-dad70ba56d4b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:25.991685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0861d31-b27b-4807-87e4-dad70ba56d4b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:25.993136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0861d31-b27b-4807-87e4-dad70ba56d4b None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application [None req-b0861d31-b27b-4807-87e4-dad70ba56d4b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 23 01:11:26.002313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 23 01:11:26.003404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.get_or_create( Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application with Lock( Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self._enter() Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application return self.creator() Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application created_value = creator( Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: ERROR keystone.server.flask.application Sep 23 01:11:26.004290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 644/1304] 10.222.0.64 () {56 vars in 1117 bytes} [Fri Sep 23 01:11:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 23 01:11:26.009468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477331b-eedf-486a-8917-4e321f6d7187 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:26.086271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:26.109241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477331b-eedf-486a-8917-4e321f6d7187 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:26.142437 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c720d9-db92-4dc6-982a-8a4987be0e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:26.143041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c720d9-db92-4dc6-982a-8a4987be0e3a None admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:26.143319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c720d9-db92-4dc6-982a-8a4987be0e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:26.143615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c720d9-db92-4dc6-982a-8a4987be0e3a None admin] PATH_INFO: `/v3/services` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:26.144285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c720d9-db92-4dc6-982a-8a4987be0e3a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:11:26.144589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c720d9-db92-4dc6-982a-8a4987be0e3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:26.145989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c720d9-db92-4dc6-982a-8a4987be0e3a None admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:26.151655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 661/1305] 10.222.0.64 () {56 vars in 1116 bytes} [Fri Sep 23 01:11:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:11:26.157973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0861d31-b27b-4807-87e4-dad70ba56d4b None admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:11:26.266253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0861d31-b27b-4807-87e4-dad70ba56d4b None admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:11:26.299882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7893c53-8249-47ca-b08d-3258cd94434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:11:26.300488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7893c53-8249-47ca-b08d-3258cd94434c None admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:11:26.300767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7893c53-8249-47ca-b08d-3258cd94434c None admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:11:26.301040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7893c53-8249-47ca-b08d-3258cd94434c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:11:26.301701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7893c53-8249-47ca-b08d-3258cd94434c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:11:26.303069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7893c53-8249-47ca-b08d-3258cd94434c None admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:11:26.561778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 645/1306] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:11:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:11:36.362669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=30 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:11:36.556647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:38.102188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f88a-79a8-4dda-8842-47e8680f8027 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:38.103002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f88a-79a8-4dda-8842-47e8680f8027 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:38.103142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420f88a-79a8-4dda-8842-47e8680f8027 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:38.106019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 662/1307] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:20:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:20:38.116228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9e7ef-bcb0-4766-8d47-f1752cc39088 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:38.117070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9e7ef-bcb0-4766-8d47-f1752cc39088 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:38.117236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9e7ef-bcb0-4766-8d47-f1752cc39088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:38.393640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-82c9e7ef-bcb0-4766-8d47-f1752cc39088 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:20:38.398432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-82c9e7ef-bcb0-4766-8d47-f1752cc39088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:38.652189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 646/1308] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 538 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:20:39.153707 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb3134-624c-4a49-8108-2037534fd89a None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:39.154650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb3134-624c-4a49-8108-2037534fd89a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:39.155093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb3134-624c-4a49-8108-2037534fd89a None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:39.157314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 663/1309] 10.222.0.64 () {54 vars in 964 bytes} [Fri Sep 23 01:20:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:20:39.165485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1061b-db91-47f0-b90c-556a7cf201a1 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:39.167308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1061b-db91-47f0-b90c-556a7cf201a1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:39.168032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1061b-db91-47f0-b90c-556a7cf201a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:39.318556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-9bf1061b-db91-47f0-b90c-556a7cf201a1 None None] MFA Rules not processed for user `3b0a7b4f607d4753bf142e1420bcd987`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:20:39.320754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9bf1061b-db91-47f0-b90c-556a7cf201a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:39.364954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:39.482736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 647/1310] 10.222.0.64 () {58 vars in 1061 bytes} [Fri Sep 23 01:20:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:20:39.484882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb3134-624c-4a49-8108-2037534fd89a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:20:39.488841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2deb3134-624c-4a49-8108-2037534fd89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:39.742711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb3134-624c-4a49-8108-2037534fd89a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:20:39.895069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e901e8-0c25-4135-9d53-178677cb8ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:20:39.895965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e901e8-0c25-4135-9d53-178677cb8ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:39.896447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e901e8-0c25-4135-9d53-178677cb8ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:39.897005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e901e8-0c25-4135-9d53-178677cb8ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:39.899580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-15e901e8-0c25-4135-9d53-178677cb8ca7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:40.025112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:40.134418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e901e8-0c25-4135-9d53-178677cb8ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:20:40.138281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e901e8-0c25-4135-9d53-178677cb8ca7 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:20:40.211070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 664/1311] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 728 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:20:43.220087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1bcd1-4758-40bd-bc64-0b4b688f0eaa None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:43.220766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1bcd1-4758-40bd-bc64-0b4b688f0eaa None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:43.220766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1bcd1-4758-40bd-bc64-0b4b688f0eaa None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:43.222055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 648/1312] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:20:43.228013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dab206-7f8c-46cc-bc96-f65940b05ef7 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:43.228580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dab206-7f8c-46cc-bc96-f65940b05ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:43.228727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dab206-7f8c-46cc-bc96-f65940b05ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:43.346250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-51dab206-7f8c-46cc-bc96-f65940b05ef7 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:20:43.348390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-51dab206-7f8c-46cc-bc96-f65940b05ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:43.491984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 665/1313] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:20:43.797718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d1981-5234-4690-a03e-d39128f5f7ea None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:43.798003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d1981-5234-4690-a03e-d39128f5f7ea None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:43.798289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d1981-5234-4690-a03e-d39128f5f7ea None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:43.799750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 649/1314] 10.222.0.64 () {54 vars in 964 bytes} [Fri Sep 23 01:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:20:43.806544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c1973-82be-43e4-b81e-f265f74c06d8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:43.806959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c1973-82be-43e4-b81e-f265f74c06d8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:43.807406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c1973-82be-43e4-b81e-f265f74c06d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:43.823081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:43.912756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-872c1973-82be-43e4-b81e-f265f74c06d8 None None] MFA Rules not processed for user `3b0a7b4f607d4753bf142e1420bcd987`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:20:43.914835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-872c1973-82be-43e4-b81e-f265f74c06d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:44.049920 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 666/1315] 10.222.0.64 () {58 vars in 1061 bytes} [Fri Sep 23 01:20:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:20:44.059811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d1981-5234-4690-a03e-d39128f5f7ea None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:20:44.061497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d31d1981-5234-4690-a03e-d39128f5f7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:44.188033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d1981-5234-4690-a03e-d39128f5f7ea None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:20:44.272596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:44.275366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448a7f5-5971-420f-ad62-0d43fa7479f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:20:44.276030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448a7f5-5971-420f-ad62-0d43fa7479f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:44.276293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448a7f5-5971-420f-ad62-0d43fa7479f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:44.276565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448a7f5-5971-420f-ad62-0d43fa7479f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:44.278252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b448a7f5-5971-420f-ad62-0d43fa7479f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:44.469059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448a7f5-5971-420f-ad62-0d43fa7479f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:20:44.471877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b448a7f5-5971-420f-ad62-0d43fa7479f9 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:20:44.507772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 650/1316] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:20:47.964889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b255401-3987-44e0-b6ae-26d967b72a8e None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:47.965457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b255401-3987-44e0-b6ae-26d967b72a8e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:47.965457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b255401-3987-44e0-b6ae-26d967b72a8e None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:47.966888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 667/1317] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:20:47.973118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10d7d6-f2e5-4b2c-8df7-a4c38e27d0d9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:47.973648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10d7d6-f2e5-4b2c-8df7-a4c38e27d0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:47.973648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10d7d6-f2e5-4b2c-8df7-a4c38e27d0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:48.054702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ae10d7d6-f2e5-4b2c-8df7-a4c38e27d0d9 None None] MFA Rules not processed for user `37677eab531b4f658648830316674597`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:20:48.056088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ae10d7d6-f2e5-4b2c-8df7-a4c38e27d0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:48.119965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:48.173481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 651/1318] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:20:47 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:20:48.474861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b255401-3987-44e0-b6ae-26d967b72a8e None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:20:48.476459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4b255401-3987-44e0-b6ae-26d967b72a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:48.590592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b255401-3987-44e0-b6ae-26d967b72a8e None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:20:48.594187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:48.670098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920a2bc-f25e-42a2-bd98-93de3f0ebcb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:20:48.670714 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a2bc-f25e-42a2-bd98-93de3f0ebcb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:48.670999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a2bc-f25e-42a2-bd98-93de3f0ebcb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:48.671276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920a2bc-f25e-42a2-bd98-93de3f0ebcb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:48.672926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f920a2bc-f25e-42a2-bd98-93de3f0ebcb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:48.806518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920a2bc-f25e-42a2-bd98-93de3f0ebcb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:20:48.808122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920a2bc-f25e-42a2-bd98-93de3f0ebcb4 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:20:48.831007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 668/1319] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:20:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:20:52.164381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ceb685-0826-44c4-a88d-3a2f1a9eae53 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:52.165154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ceb685-0826-44c4-a88d-3a2f1a9eae53 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:52.165154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ceb685-0826-44c4-a88d-3a2f1a9eae53 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:52.166375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 652/1320] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:20:52.173068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9bd6f-2236-45f2-8c75-ba5af01c0fd8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:52.173623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9bd6f-2236-45f2-8c75-ba5af01c0fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:52.173623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9bd6f-2236-45f2-8c75-ba5af01c0fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:52.222802 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:52.249045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-e7d9bd6f-2236-45f2-8c75-ba5af01c0fd8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:20:52.252168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e7d9bd6f-2236-45f2-8c75-ba5af01c0fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:52.358404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 669/1321] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:20:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:20:52.640394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ceb685-0826-44c4-a88d-3a2f1a9eae53 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:20:52.642000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-42ceb685-0826-44c4-a88d-3a2f1a9eae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:52.758032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ceb685-0826-44c4-a88d-3a2f1a9eae53 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:20:52.837340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8e742-0f67-4b49-81f9-d2a79d1ef16f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:20:52.837949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8e742-0f67-4b49-81f9-d2a79d1ef16f service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:52.838228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8e742-0f67-4b49-81f9-d2a79d1ef16f service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:52.838503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8e742-0f67-4b49-81f9-d2a79d1ef16f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:52.840204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3bc8e742-0f67-4b49-81f9-d2a79d1ef16f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:52.847879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:52.958732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8e742-0f67-4b49-81f9-d2a79d1ef16f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:20:52.960148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8e742-0f67-4b49-81f9-d2a79d1ef16f service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:20:52.979822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 653/1322] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:20:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:20:56.388187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71450f-dd4d-41d9-b32f-e4e0a8e1dc9d None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:56.388837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71450f-dd4d-41d9-b32f-e4e0a8e1dc9d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:56.388837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71450f-dd4d-41d9-b32f-e4e0a8e1dc9d None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:56.390349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 670/1323] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:20:56.397579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900daa7d-3881-4096-b7ac-50672676e4fd None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:56.398100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900daa7d-3881-4096-b7ac-50672676e4fd None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:56.398570 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900daa7d-3881-4096-b7ac-50672676e4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:56.482844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-900daa7d-3881-4096-b7ac-50672676e4fd None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:20:56.484711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-900daa7d-3881-4096-b7ac-50672676e4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:56.549393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:56.591847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 654/1324] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:20:56.887742 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71450f-dd4d-41d9-b32f-e4e0a8e1dc9d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:20:56.889325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7c71450f-dd4d-41d9-b32f-e4e0a8e1dc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:57.023212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71450f-dd4d-41d9-b32f-e4e0a8e1dc9d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:20:57.038951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:57.112785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307a51a-e60c-44a0-bcb8-5ea1120dd560 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:20:57.113405 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307a51a-e60c-44a0-bcb8-5ea1120dd560 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:57.113692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307a51a-e60c-44a0-bcb8-5ea1120dd560 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:57.113969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307a51a-e60c-44a0-bcb8-5ea1120dd560 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:57.115722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c307a51a-e60c-44a0-bcb8-5ea1120dd560 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:57.246293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307a51a-e60c-44a0-bcb8-5ea1120dd560 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:20:57.247518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307a51a-e60c-44a0-bcb8-5ea1120dd560 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:20:57.268979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 671/1325] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:20:59.621873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7702d-7001-4cd1-a8c3-f7c3ccd5c942 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:59.622473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7702d-7001-4cd1-a8c3-f7c3ccd5c942 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:59.622473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7702d-7001-4cd1-a8c3-f7c3ccd5c942 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:59.624066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 655/1326] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:20:59.630784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09afc15-903f-4180-9d0c-332dddfddb7f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:20:59.631449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09afc15-903f-4180-9d0c-332dddfddb7f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:20:59.631449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09afc15-903f-4180-9d0c-332dddfddb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:20:59.698064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:20:59.719804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f09afc15-903f-4180-9d0c-332dddfddb7f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:20:59.721070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f09afc15-903f-4180-9d0c-332dddfddb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:20:59.866049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 672/1327] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:00.172172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7702d-7001-4cd1-a8c3-f7c3ccd5c942 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:21:00.173935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-45c7702d-7001-4cd1-a8c3-f7c3ccd5c942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:00.311273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7702d-7001-4cd1-a8c3-f7c3ccd5c942 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:21:00.400175 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd61bfeb-50c2-4d7a-9a83-4fd363a43519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:21:00.400896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61bfeb-50c2-4d7a-9a83-4fd363a43519 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:00.401177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61bfeb-50c2-4d7a-9a83-4fd363a43519 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:00.401456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd61bfeb-50c2-4d7a-9a83-4fd363a43519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:00.403137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-dd61bfeb-50c2-4d7a-9a83-4fd363a43519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:00.424577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:00.544564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61bfeb-50c2-4d7a-9a83-4fd363a43519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:21:00.546106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd61bfeb-50c2-4d7a-9a83-4fd363a43519 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:21:00.573857 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 656/1328] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:21:03.132340 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1a5ea-5b8c-48ae-88b5-e47dd5b52b37 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:03.132961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1a5ea-5b8c-48ae-88b5-e47dd5b52b37 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:03.133137 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1a5ea-5b8c-48ae-88b5-e47dd5b52b37 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:03.135630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 673/1329] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:21:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:03.142739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698670ed-819f-4797-b041-f9cb2f962448 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:03.143518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698670ed-819f-4797-b041-f9cb2f962448 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:03.143518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698670ed-819f-4797-b041-f9cb2f962448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:03.230875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-698670ed-819f-4797-b041-f9cb2f962448 None None] MFA Rules not processed for user `37677eab531b4f658648830316674597`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:03.232069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-698670ed-819f-4797-b041-f9cb2f962448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:03.316915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:03.359762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 657/1330] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:21:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:03.666674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1a5ea-5b8c-48ae-88b5-e47dd5b52b37 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:21:03.668302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-cde1a5ea-5b8c-48ae-88b5-e47dd5b52b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:03.863681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1a5ea-5b8c-48ae-88b5-e47dd5b52b37 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:21:03.890744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:03.965194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867230e-9cc6-4cab-9d7d-5a64fd73c1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:21:03.965907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867230e-9cc6-4cab-9d7d-5a64fd73c1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:03.966187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867230e-9cc6-4cab-9d7d-5a64fd73c1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:03.966462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867230e-9cc6-4cab-9d7d-5a64fd73c1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:03.968716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d867230e-9cc6-4cab-9d7d-5a64fd73c1c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:04.151710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867230e-9cc6-4cab-9d7d-5a64fd73c1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:21:04.154023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867230e-9cc6-4cab-9d7d-5a64fd73c1c1 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:21:04.191814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 674/1331] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:21:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:21:07.201497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52195-2b0b-40f4-a725-518d9da02a20 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:07.202731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52195-2b0b-40f4-a725-518d9da02a20 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:07.202731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa52195-2b0b-40f4-a725-518d9da02a20 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:07.204339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 658/1332] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:07.210962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619ea1d-e18a-437d-8a0a-70ef0dfe192e None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:07.211507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619ea1d-e18a-437d-8a0a-70ef0dfe192e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:07.211662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619ea1d-e18a-437d-8a0a-70ef0dfe192e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:07.284581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:07.287368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-4619ea1d-e18a-437d-8a0a-70ef0dfe192e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:07.288640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4619ea1d-e18a-437d-8a0a-70ef0dfe192e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:07.393296 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 675/1333] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:07.680842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52195-2b0b-40f4-a725-518d9da02a20 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:21:07.689040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3fa52195-2b0b-40f4-a725-518d9da02a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:07.826578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa52195-2b0b-40f4-a725-518d9da02a20 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:21:07.935062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47744c5-ceca-4a54-a70c-9bb8382dfd64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:21:07.935631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47744c5-ceca-4a54-a70c-9bb8382dfd64 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:07.935916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47744c5-ceca-4a54-a70c-9bb8382dfd64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:07.936194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47744c5-ceca-4a54-a70c-9bb8382dfd64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:07.938504 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f47744c5-ceca-4a54-a70c-9bb8382dfd64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:07.971569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:08.067696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47744c5-ceca-4a54-a70c-9bb8382dfd64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:21:08.069165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47744c5-ceca-4a54-a70c-9bb8382dfd64 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:21:08.091308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 659/1334] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:21:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:21:10.392170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a48dfe-df81-48f8-8a32-0585e29ebd74 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:10.392721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a48dfe-df81-48f8-8a32-0585e29ebd74 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:10.392849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a48dfe-df81-48f8-8a32-0585e29ebd74 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:10.394189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 676/1335] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:10.400668 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625a9c9-ca6f-4a69-87f8-d18fd1c39c94 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:10.401202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625a9c9-ca6f-4a69-87f8-d18fd1c39c94 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:10.401202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625a9c9-ca6f-4a69-87f8-d18fd1c39c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:10.479051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b625a9c9-ca6f-4a69-87f8-d18fd1c39c94 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:10.480338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b625a9c9-ca6f-4a69-87f8-d18fd1c39c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:10.560483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:10.587465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 660/1336] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:10.882986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a48dfe-df81-48f8-8a32-0585e29ebd74 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:21:10.884576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e9a48dfe-df81-48f8-8a32-0585e29ebd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:11.011262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a48dfe-df81-48f8-8a32-0585e29ebd74 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:21:11.042453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:11.091026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b30e18-64e8-419f-8bb4-7e17a65402e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:21:11.091606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b30e18-64e8-419f-8bb4-7e17a65402e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:11.091968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b30e18-64e8-419f-8bb4-7e17a65402e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:11.092251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b30e18-64e8-419f-8bb4-7e17a65402e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:11.093916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-43b30e18-64e8-419f-8bb4-7e17a65402e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:11.212766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b30e18-64e8-419f-8bb4-7e17a65402e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:21:11.214239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b30e18-64e8-419f-8bb4-7e17a65402e5 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:21:11.241692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 677/1337] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:21:14.206078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20709b19-cad5-4331-9a68-ecf72a456747 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:14.206701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20709b19-cad5-4331-9a68-ecf72a456747 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:14.206701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20709b19-cad5-4331-9a68-ecf72a456747 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:14.208084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 661/1338] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:14.213857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d60ec-8dee-451a-a9c4-8a891177f421 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:14.214383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d60ec-8dee-451a-a9c4-8a891177f421 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:14.214383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d60ec-8dee-451a-a9c4-8a891177f421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:14.305303 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9b0d60ec-8dee-451a-a9c4-8a891177f421 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:14.307462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9b0d60ec-8dee-451a-a9c4-8a891177f421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:14.323645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:14.440307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 678/1339] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:21:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:14.727965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20709b19-cad5-4331-9a68-ecf72a456747 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:21:14.729529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-20709b19-cad5-4331-9a68-ecf72a456747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:14.849384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20709b19-cad5-4331-9a68-ecf72a456747 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:21:14.953929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9aaf26-f8c9-4d3b-b60f-3d22f9054485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:21:14.954825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9aaf26-f8c9-4d3b-b60f-3d22f9054485 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:14.955837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9aaf26-f8c9-4d3b-b60f-3d22f9054485 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:14.956186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9aaf26-f8c9-4d3b-b60f-3d22f9054485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:14.959396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0b9aaf26-f8c9-4d3b-b60f-3d22f9054485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:15.009750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:15.095857 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9aaf26-f8c9-4d3b-b60f-3d22f9054485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:21:15.097328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9aaf26-f8c9-4d3b-b60f-3d22f9054485 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:21:15.119070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 662/1340] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:21:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:21:20.723453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160246ff-45d0-4cd4-b27b-a259dc158042 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:20.724292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160246ff-45d0-4cd4-b27b-a259dc158042 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:20.724292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160246ff-45d0-4cd4-b27b-a259dc158042 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:20.725723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 679/1341] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:21:20.732172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aa03d-8d9c-44f9-b764-e7cd24abaeea None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:20.732999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aa03d-8d9c-44f9-b764-e7cd24abaeea None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:20.732999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135aa03d-8d9c-44f9-b764-e7cd24abaeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:20.862896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-135aa03d-8d9c-44f9-b764-e7cd24abaeea None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:20.864622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-135aa03d-8d9c-44f9-b764-e7cd24abaeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:21.039663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:21.063409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 663/1342] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:21:21.595248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160246ff-45d0-4cd4-b27b-a259dc158042 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:21:21.598136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-160246ff-45d0-4cd4-b27b-a259dc158042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:21.763692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160246ff-45d0-4cd4-b27b-a259dc158042 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:21:21.831655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:21.899869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6bcfc-ae33-46fc-bbb1-109fd9892582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:21:21.900912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6bcfc-ae33-46fc-bbb1-109fd9892582 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:21.901243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6bcfc-ae33-46fc-bbb1-109fd9892582 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:21.901638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6bcfc-ae33-46fc-bbb1-109fd9892582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:21.904314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f7f6bcfc-ae33-46fc-bbb1-109fd9892582 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:22.104028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6bcfc-ae33-46fc-bbb1-109fd9892582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:21:22.105927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6bcfc-ae33-46fc-bbb1-109fd9892582 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:21:22.136369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 680/1343] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:21:26.688567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28f3bd-c220-407d-99d9-50c92f7dc5f2 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:26.689364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28f3bd-c220-407d-99d9-50c92f7dc5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:26.689364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28f3bd-c220-407d-99d9-50c92f7dc5f2 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:26.690640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 664/1344] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:26.697996 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f75ba-1e9b-463d-8682-9ae15d2c607b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:26.698634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f75ba-1e9b-463d-8682-9ae15d2c607b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:26.698915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f75ba-1e9b-463d-8682-9ae15d2c607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:26.806968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-972f75ba-1e9b-463d-8682-9ae15d2c607b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:26.809401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-972f75ba-1e9b-463d-8682-9ae15d2c607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:26.850292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:27.009086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 681/1345] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:21:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:27.344012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28f3bd-c220-407d-99d9-50c92f7dc5f2 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:21:27.346917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5b28f3bd-c220-407d-99d9-50c92f7dc5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:27.575224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28f3bd-c220-407d-99d9-50c92f7dc5f2 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:21:27.716118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a060eb-66d5-4d87-937a-6cd92c87a60f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:21:27.716827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a060eb-66d5-4d87-937a-6cd92c87a60f service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:27.717531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a060eb-66d5-4d87-937a-6cd92c87a60f service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:27.717810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a060eb-66d5-4d87-937a-6cd92c87a60f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:27.719625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-16a060eb-66d5-4d87-937a-6cd92c87a60f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:27.801877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:27.894946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a060eb-66d5-4d87-937a-6cd92c87a60f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:21:27.896649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a060eb-66d5-4d87-937a-6cd92c87a60f service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:21:27.929696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 665/1346] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:21:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:21:33.296385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0794b-6ef9-4696-832d-6d2af54123bf None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:33.297218 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0794b-6ef9-4696-832d-6d2af54123bf None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:33.297470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0794b-6ef9-4696-832d-6d2af54123bf None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:33.299235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 682/1347] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:21:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:21:33.307000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2ac7-6dde-4266-aa93-a156f0b055c0 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:33.307677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2ac7-6dde-4266-aa93-a156f0b055c0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:33.308088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939c2ac7-6dde-4266-aa93-a156f0b055c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:33.406073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-939c2ac7-6dde-4266-aa93-a156f0b055c0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:33.407427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-939c2ac7-6dde-4266-aa93-a156f0b055c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:33.575737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:33.579435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 666/1348] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:33.895590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0794b-6ef9-4696-832d-6d2af54123bf None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:21:33.897522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7ab0794b-6ef9-4696-832d-6d2af54123bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:34.030957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0794b-6ef9-4696-832d-6d2af54123bf None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:21:34.096963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:34.132823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bd2cf-b557-4b7a-af72-46c684ae7008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:21:34.133593 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bd2cf-b557-4b7a-af72-46c684ae7008 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:34.133900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bd2cf-b557-4b7a-af72-46c684ae7008 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:34.134247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bd2cf-b557-4b7a-af72-46c684ae7008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:34.136835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-465bd2cf-b557-4b7a-af72-46c684ae7008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:34.281905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bd2cf-b557-4b7a-af72-46c684ae7008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:21:34.283758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bd2cf-b557-4b7a-af72-46c684ae7008 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:21:34.308599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 683/1349] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:21:40.682726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2ac35-81ca-432c-bedb-77251992635f None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:40.688453 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2ac35-81ca-432c-bedb-77251992635f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:40.688453 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2ac35-81ca-432c-bedb-77251992635f None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:40.688453 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 667/1350] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:21:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:40.694448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fe3dd-bc73-4efe-a527-a9ff29423331 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:40.695244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fe3dd-bc73-4efe-a527-a9ff29423331 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:40.695244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fe3dd-bc73-4efe-a527-a9ff29423331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:40.808187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-402fe3dd-bc73-4efe-a527-a9ff29423331 None None] MFA Rules not processed for user `9f73bca3896440258430cfce042faa42`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:40.809860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-402fe3dd-bc73-4efe-a527-a9ff29423331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:40.837647 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:41.032216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 684/1351] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:21:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:48.080471 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48af27e-9ac9-4346-88eb-aa5c7cf3a292 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:48.082164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48af27e-9ac9-4346-88eb-aa5c7cf3a292 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:48.082835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48af27e-9ac9-4346-88eb-aa5c7cf3a292 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:48.086345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 668/1352] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:21:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:48.093198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7d995-8beb-49a3-9022-3cc2b0d9eabd None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:48.093792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7d995-8beb-49a3-9022-3cc2b0d9eabd None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:48.094356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7d995-8beb-49a3-9022-3cc2b0d9eabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:48.240891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-54b7d995-8beb-49a3-9022-3cc2b0d9eabd None None] MFA Rules not processed for user `9f73bca3896440258430cfce042faa42`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:48.242226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-54b7d995-8beb-49a3-9022-3cc2b0d9eabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:48.361318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:48.443622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 685/1353] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:21:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:49.987377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422ffc8-0d26-4529-a2dc-5b9c6961e110 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:49.988008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422ffc8-0d26-4529-a2dc-5b9c6961e110 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:49.988336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422ffc8-0d26-4529-a2dc-5b9c6961e110 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:49.991019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 669/1354] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:21:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:49.999397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65045f-5796-422b-b447-e39e01e0166d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:50.000182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65045f-5796-422b-b447-e39e01e0166d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:50.000421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65045f-5796-422b-b447-e39e01e0166d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:50.115883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-fe65045f-5796-422b-b447-e39e01e0166d None None] MFA Rules not processed for user `9f73bca3896440258430cfce042faa42`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:50.117621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-fe65045f-5796-422b-b447-e39e01e0166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:50.300057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 686/1355] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:21:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:54.254278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db0e7c-9c41-4f46-b2c9-dfb92ae9e743 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:54.254803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db0e7c-9c41-4f46-b2c9-dfb92ae9e743 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:54.255065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db0e7c-9c41-4f46-b2c9-dfb92ae9e743 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:54.262576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 670/1356] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:21:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:54.267121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd5ba-1397-4689-8a05-a10474223f64 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:54.267673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd5ba-1397-4689-8a05-a10474223f64 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:54.267787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd5ba-1397-4689-8a05-a10474223f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:54.319749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:54.398163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a65cd5ba-1397-4689-8a05-a10474223f64 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:54.399732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a65cd5ba-1397-4689-8a05-a10474223f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:54.595869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 687/1357] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:54.936812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63e269-bb1e-41d8-aba5-23924db8a19e None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:54.937489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63e269-bb1e-41d8-aba5-23924db8a19e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:54.937993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63e269-bb1e-41d8-aba5-23924db8a19e None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:54.940175 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 671/1358] 10.222.0.64 () {54 vars in 961 bytes} [Fri Sep 23 01:21:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:54.947858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fa515-12e6-47ef-85b6-b4b23ad18a4c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:54.951397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fa515-12e6-47ef-85b6-b4b23ad18a4c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:54.951397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fa515-12e6-47ef-85b6-b4b23ad18a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:55.069814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-0a1fa515-12e6-47ef-85b6-b4b23ad18a4c None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:55.071526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0a1fa515-12e6-47ef-85b6-b4b23ad18a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:55.146855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:55.259086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 688/1359] 10.222.0.64 () {58 vars in 1058 bytes} [Fri Sep 23 01:21:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:21:55.269762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63e269-bb1e-41d8-aba5-23924db8a19e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:21:55.275343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9b63e269-bb1e-41d8-aba5-23924db8a19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:55.553819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63e269-bb1e-41d8-aba5-23924db8a19e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:21:55.679741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:55.729394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b375b8-f623-4af8-bbc6-0d362a9b0c86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:21:55.730190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b375b8-f623-4af8-bbc6-0d362a9b0c86 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:55.730519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b375b8-f623-4af8-bbc6-0d362a9b0c86 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:55.730801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b375b8-f623-4af8-bbc6-0d362a9b0c86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:55.732692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-18b375b8-f623-4af8-bbc6-0d362a9b0c86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:55.878648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b375b8-f623-4af8-bbc6-0d362a9b0c86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:21:55.880804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b375b8-f623-4af8-bbc6-0d362a9b0c86 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:21:55.916058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 672/1360] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:21:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:21:59.126645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fe18e-0f1d-45a2-8149-f37b28ad5bb6 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:59.127505 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fe18e-0f1d-45a2-8149-f37b28ad5bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:59.128304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fe18e-0f1d-45a2-8149-f37b28ad5bb6 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:59.131610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 689/1361] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:21:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:21:59.142188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e03f1-e269-4403-bb5f-8194102e33e2 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:21:59.142951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e03f1-e269-4403-bb5f-8194102e33e2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:21:59.143097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e03f1-e269-4403-bb5f-8194102e33e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:21:59.280654 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-943e03f1-e269-4403-bb5f-8194102e33e2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:21:59.282819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-943e03f1-e269-4403-bb5f-8194102e33e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:21:59.365957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:21:59.499823 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 673/1362] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:00.016508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fe18e-0f1d-45a2-8149-f37b28ad5bb6 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:00.018621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6a0fe18e-0f1d-45a2-8149-f37b28ad5bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:00.253924 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fe18e-0f1d-45a2-8149-f37b28ad5bb6 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:00.270404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:00.484501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e9bdc-b29f-4d91-9348-342b90fb84fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:00.485519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e9bdc-b29f-4d91-9348-342b90fb84fb service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:00.485942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e9bdc-b29f-4d91-9348-342b90fb84fb service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:00.486348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e9bdc-b29f-4d91-9348-342b90fb84fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:00.489274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-951e9bdc-b29f-4d91-9348-342b90fb84fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:00.651420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e9bdc-b29f-4d91-9348-342b90fb84fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:00.654909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e9bdc-b29f-4d91-9348-342b90fb84fb service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:00.688308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 690/1363] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:04.070945 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f6af-f8af-4ce6-b5b6-2e162bd1cc25 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:04.071501 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f6af-f8af-4ce6-b5b6-2e162bd1cc25 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:04.071501 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f6af-f8af-4ce6-b5b6-2e162bd1cc25 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:04.073395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 674/1364] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:04.080028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8de27e-e7de-4104-8182-f50b76cee345 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:04.080983 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8de27e-e7de-4104-8182-f50b76cee345 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:04.080983 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8de27e-e7de-4104-8182-f50b76cee345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:04.101299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:04.173259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-7b8de27e-e7de-4104-8182-f50b76cee345 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:04.174628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7b8de27e-e7de-4104-8182-f50b76cee345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:04.404360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 691/1365] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:04.777098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e72410-d516-431b-b8ad-f018c7f3f854 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:04.777593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e72410-d516-431b-b8ad-f018c7f3f854 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:04.778123 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e72410-d516-431b-b8ad-f018c7f3f854 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:04.781437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 675/1366] 10.222.0.64 () {54 vars in 961 bytes} [Fri Sep 23 01:22:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:04.787858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec12ea-7228-4029-8a5f-91b3cdb32579 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:04.788356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec12ea-7228-4029-8a5f-91b3cdb32579 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:04.788965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec12ea-7228-4029-8a5f-91b3cdb32579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:04.932687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-1aec12ea-7228-4029-8a5f-91b3cdb32579 None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:04.934100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1aec12ea-7228-4029-8a5f-91b3cdb32579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:04.983516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67610d64-d673-4e41-b786-eb26e27b7fab None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:04.983929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67610d64-d673-4e41-b786-eb26e27b7fab None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:04.984148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67610d64-d673-4e41-b786-eb26e27b7fab None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:04.985569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 676/1367] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:22:04.985961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:04.991996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84011be-7a17-45f9-b468-276f180a409d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:04.991996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84011be-7a17-45f9-b468-276f180a409d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:04.992286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84011be-7a17-45f9-b468-276f180a409d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:05.099360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a84011be-7a17-45f9-b468-276f180a409d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:05.101644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a84011be-7a17-45f9-b468-276f180a409d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:05.121728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 692/1368] 10.222.0.64 () {58 vars in 1058 bytes} [Fri Sep 23 01:22:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:05.130944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec12ea-7228-4029-8a5f-91b3cdb32579 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:05.152429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec12ea-7228-4029-8a5f-91b3cdb32579 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:05.261686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 677/1369] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:22:05.329456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf03e76b-20e4-42c6-bd9a-7b3700b65de4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:05.330650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03e76b-20e4-42c6-bd9a-7b3700b65de4 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:05.330947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03e76b-20e4-42c6-bd9a-7b3700b65de4 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:05.331314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf03e76b-20e4-42c6-bd9a-7b3700b65de4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:05.333131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-cf03e76b-20e4-42c6-bd9a-7b3700b65de4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:05.359906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:05.508814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03e76b-20e4-42c6-bd9a-7b3700b65de4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:05.512026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf03e76b-20e4-42c6-bd9a-7b3700b65de4 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:05.555874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 693/1370] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:05.646711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab2ac5-fffa-425c-80d3-925bfa001171 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:05.646711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab2ac5-fffa-425c-80d3-925bfa001171 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:05.646711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab2ac5-fffa-425c-80d3-925bfa001171 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:05.648244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 678/1371] 10.222.0.64 () {54 vars in 963 bytes} [Fri Sep 23 01:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:05.660031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ac9b5-bdfa-41c5-90e6-a5e52ef6c2d2 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:05.660394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ac9b5-bdfa-41c5-90e6-a5e52ef6c2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:05.660602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ac9b5-bdfa-41c5-90e6-a5e52ef6c2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:05.747524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-680ac9b5-bdfa-41c5-90e6-a5e52ef6c2d2 None None] MFA Rules not processed for user `51ae9d865bb4442490ca61b18d1f7ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:05.748802 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-680ac9b5-bdfa-41c5-90e6-a5e52ef6c2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:05.850284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 694/1372] 10.222.0.64 () {58 vars in 1060 bytes} [Fri Sep 23 01:22:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:05.859205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab2ac5-fffa-425c-80d3-925bfa001171 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:05.861060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4cab2ac5-fffa-425c-80d3-925bfa001171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:05.884142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:06.031385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab2ac5-fffa-425c-80d3-925bfa001171 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:06.190911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524617e-0a46-460f-8f86-5e98f21685e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:06.192176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524617e-0a46-460f-8f86-5e98f21685e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:06.192500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524617e-0a46-460f-8f86-5e98f21685e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:06.192971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524617e-0a46-460f-8f86-5e98f21685e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:06.196399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0524617e-0a46-460f-8f86-5e98f21685e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:06.390698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:06.493577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524617e-0a46-460f-8f86-5e98f21685e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:06.497931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524617e-0a46-460f-8f86-5e98f21685e8 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:06.547037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 679/1373] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:08.973944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc430f3-d33c-486c-95be-d68d0921a7b4 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:08.974768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc430f3-d33c-486c-95be-d68d0921a7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:08.974938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc430f3-d33c-486c-95be-d68d0921a7b4 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:08.977721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 695/1374] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:08.987527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70bb3c-9dda-4239-b9d5-518992f06bb9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:08.988463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70bb3c-9dda-4239-b9d5-518992f06bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:08.988634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70bb3c-9dda-4239-b9d5-518992f06bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:09.107376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ab70bb3c-9dda-4239-b9d5-518992f06bb9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:09.109782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ab70bb3c-9dda-4239-b9d5-518992f06bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:09.269086 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 680/1375] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:09.645874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc430f3-d33c-486c-95be-d68d0921a7b4 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:09.648161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6bc430f3-d33c-486c-95be-d68d0921a7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:09.660134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:09.904886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc430f3-d33c-486c-95be-d68d0921a7b4 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:10.050448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:10.109846 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e03358-2f3c-4a77-9354-88fbc5a2e838 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:10.110604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03358-2f3c-4a77-9354-88fbc5a2e838 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:10.110955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03358-2f3c-4a77-9354-88fbc5a2e838 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:10.111297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03358-2f3c-4a77-9354-88fbc5a2e838 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:10.113093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c8e03358-2f3c-4a77-9354-88fbc5a2e838 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:10.309463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e03358-2f3c-4a77-9354-88fbc5a2e838 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:10.312891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e03358-2f3c-4a77-9354-88fbc5a2e838 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:10.363716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 696/1376] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:13.926021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1755706-451e-49bf-b537-b979ddfe8b24 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:13.926998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1755706-451e-49bf-b537-b979ddfe8b24 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:13.927179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1755706-451e-49bf-b537-b979ddfe8b24 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:13.929829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 681/1377] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:13.941683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561561-026e-4811-880d-23f1cf75fc41 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:13.943138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561561-026e-4811-880d-23f1cf75fc41 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:13.943662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561561-026e-4811-880d-23f1cf75fc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:14.082248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-4a561561-026e-4811-880d-23f1cf75fc41 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:14.084474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4a561561-026e-4811-880d-23f1cf75fc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:14.180584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:14.309290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 697/1378] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:14.859611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1755706-451e-49bf-b537-b979ddfe8b24 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:14.863341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b1755706-451e-49bf-b537-b979ddfe8b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:14.919361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:15.156907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1755706-451e-49bf-b537-b979ddfe8b24 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:15.364984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:15.390090 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03244823-8956-4c76-a549-9dd0ae2deaca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:15.390973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03244823-8956-4c76-a549-9dd0ae2deaca service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:15.391401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03244823-8956-4c76-a549-9dd0ae2deaca service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:15.391872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03244823-8956-4c76-a549-9dd0ae2deaca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:15.393917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-03244823-8956-4c76-a549-9dd0ae2deaca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:15.569229 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03244823-8956-4c76-a549-9dd0ae2deaca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:15.572677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03244823-8956-4c76-a549-9dd0ae2deaca service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:15.598866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 682/1379] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:19.284880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88c085-2b8d-4df2-9658-1bca5c7ac24f None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:19.285648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88c085-2b8d-4df2-9658-1bca5c7ac24f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:19.286276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88c085-2b8d-4df2-9658-1bca5c7ac24f None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:19.290348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 698/1380] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:19.300402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776be6f-750e-46f9-80dc-292dd23a423e None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:19.301637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776be6f-750e-46f9-80dc-292dd23a423e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:19.301637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776be6f-750e-46f9-80dc-292dd23a423e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:19.436205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-4776be6f-750e-46f9-80dc-292dd23a423e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:19.437486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4776be6f-750e-46f9-80dc-292dd23a423e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:19.526378 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:19.586080 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 683/1381] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:19.956387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88c085-2b8d-4df2-9658-1bca5c7ac24f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:19.958281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0e88c085-2b8d-4df2-9658-1bca5c7ac24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:20.261483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88c085-2b8d-4df2-9658-1bca5c7ac24f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:20.312896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:20.558698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3668f013-2a2c-41e9-b0ce-afc21b12bc07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:20.560049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668f013-2a2c-41e9-b0ce-afc21b12bc07 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:20.560438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668f013-2a2c-41e9-b0ce-afc21b12bc07 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:20.560935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3668f013-2a2c-41e9-b0ce-afc21b12bc07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:20.563748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3668f013-2a2c-41e9-b0ce-afc21b12bc07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:20.770947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668f013-2a2c-41e9-b0ce-afc21b12bc07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:20.774224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3668f013-2a2c-41e9-b0ce-afc21b12bc07 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:20.809993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 699/1382] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 853 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:24.501637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afa573-e6ee-4059-9fda-f0ee6a6e4aef None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:24.502268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afa573-e6ee-4059-9fda-f0ee6a6e4aef None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:24.502268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afa573-e6ee-4059-9fda-f0ee6a6e4aef None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:24.503749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 684/1383] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:24.513289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8718da-9f15-4803-bb80-8b8dfbb9afb2 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:24.513289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8718da-9f15-4803-bb80-8b8dfbb9afb2 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:24.513289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8718da-9f15-4803-bb80-8b8dfbb9afb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:24.563098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:24.600797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3a8718da-9f15-4803-bb80-8b8dfbb9afb2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:24.602257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3a8718da-9f15-4803-bb80-8b8dfbb9afb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:24.716209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 700/1384] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:25.032982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afa573-e6ee-4059-9fda-f0ee6a6e4aef None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:25.034990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f2afa573-e6ee-4059-9fda-f0ee6a6e4aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:25.235345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afa573-e6ee-4059-9fda-f0ee6a6e4aef None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:25.295915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:25.388645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6c67-c999-4401-85dc-71c16cd08ff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:25.389635 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6c67-c999-4401-85dc-71c16cd08ff2 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:25.389908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6c67-c999-4401-85dc-71c16cd08ff2 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:25.390186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6c67-c999-4401-85dc-71c16cd08ff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:25.391964 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b9fa6c67-c999-4401-85dc-71c16cd08ff2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:25.542876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6c67-c999-4401-85dc-71c16cd08ff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:25.545081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6c67-c999-4401-85dc-71c16cd08ff2 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:25.565645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 685/1385] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:27.085926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd2797-936f-43d3-b289-4ccd40ff602f None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:27.086730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd2797-936f-43d3-b289-4ccd40ff602f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:27.086730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd2797-936f-43d3-b289-4ccd40ff602f None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:27.089171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 701/1386] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:27.095736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395084f5-30bf-491a-9cae-3ccefb44f38c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:27.096319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395084f5-30bf-491a-9cae-3ccefb44f38c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:27.096537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395084f5-30bf-491a-9cae-3ccefb44f38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:27.195987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-395084f5-30bf-491a-9cae-3ccefb44f38c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:27.197299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-395084f5-30bf-491a-9cae-3ccefb44f38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:27.199753 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:27.365891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 686/1387] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:27.685110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961983f2-3fa2-4975-be93-c3d9973d8bd1 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:27.685485 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961983f2-3fa2-4975-be93-c3d9973d8bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:27.685837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961983f2-3fa2-4975-be93-c3d9973d8bd1 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:27.687338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 702/1388] 10.222.0.64 () {54 vars in 962 bytes} [Fri Sep 23 01:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:27.694143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f4e64-6741-4f05-ba5c-abe33c38aeda None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:27.694424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f4e64-6741-4f05-ba5c-abe33c38aeda None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:27.694793 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f4e64-6741-4f05-ba5c-abe33c38aeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:27.815339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-fe3f4e64-6741-4f05-ba5c-abe33c38aeda None None] MFA Rules not processed for user `6f10c258fe5f45ccab1669fa32ed6029`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:27.817431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fe3f4e64-6741-4f05-ba5c-abe33c38aeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:27.967795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:27.973341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 687/1389] 10.222.0.64 () {58 vars in 1059 bytes} [Fri Sep 23 01:22:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:27.985550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961983f2-3fa2-4975-be93-c3d9973d8bd1 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:27.987492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-961983f2-3fa2-4975-be93-c3d9973d8bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:28.025832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e6150-ab8e-4491-a084-dd262bd9ef02 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:28.027456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e6150-ab8e-4491-a084-dd262bd9ef02 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:28.027456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e6150-ab8e-4491-a084-dd262bd9ef02 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:28.030200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 688/1390] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:22:28.039686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148cd84-7a49-4a3c-9948-637b239dd356 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:28.040188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148cd84-7a49-4a3c-9948-637b239dd356 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:28.040690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d148cd84-7a49-4a3c-9948-637b239dd356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:28.140880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d148cd84-7a49-4a3c-9948-637b239dd356 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:28.142829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d148cd84-7a49-4a3c-9948-637b239dd356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:28.143015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961983f2-3fa2-4975-be93-c3d9973d8bd1 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:28.171134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:28.272979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e408dc74-03af-4ea2-89d2-2001698f1dcc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f10c258fe5f45ccab1669fa32ed6029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:28.273969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408dc74-03af-4ea2-89d2-2001698f1dcc service glance] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:28.274465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408dc74-03af-4ea2-89d2-2001698f1dcc service glance] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:28.274927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e408dc74-03af-4ea2-89d2-2001698f1dcc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:28.277512 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e408dc74-03af-4ea2-89d2-2001698f1dcc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:28.297256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 689/1391] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:22:28.462812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408dc74-03af-4ea2-89d2-2001698f1dcc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:28.464337 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e408dc74-03af-4ea2-89d2-2001698f1dcc service glance] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:28.487662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 703/1392] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:22:28.617245 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148cd84-7a49-4a3c-9948-637b239dd356 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:28.619750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d148cd84-7a49-4a3c-9948-637b239dd356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:28.761286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:28.831950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d148cd84-7a49-4a3c-9948-637b239dd356 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:29.025177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54272bab-b49e-4fa1-a45d-03674eeeac25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:29.026034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54272bab-b49e-4fa1-a45d-03674eeeac25 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:29.026530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54272bab-b49e-4fa1-a45d-03674eeeac25 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:29.026985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54272bab-b49e-4fa1-a45d-03674eeeac25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:29.030070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-54272bab-b49e-4fa1-a45d-03674eeeac25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:29.140214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:29.164912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54272bab-b49e-4fa1-a45d-03674eeeac25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:29.166568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54272bab-b49e-4fa1-a45d-03674eeeac25 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:29.187894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 690/1393] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:29.579493 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b722b46-b4a5-4491-8ce4-8426d67a8b1d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:29.580160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b722b46-b4a5-4491-8ce4-8426d67a8b1d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:29.580160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b722b46-b4a5-4491-8ce4-8426d67a8b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:29.647239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:29.669494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-0b722b46-b4a5-4491-8ce4-8426d67a8b1d None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:29.670747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0b722b46-b4a5-4491-8ce4-8426d67a8b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:29.778257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 704/1394] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:22:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:29.833237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35827a6-4894-41eb-bb9d-8907895de8ef None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:29.833522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35827a6-4894-41eb-bb9d-8907895de8ef None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:29.833800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35827a6-4894-41eb-bb9d-8907895de8ef None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:29.835460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 691/1395] 10.222.0.64 () {54 vars in 943 bytes} [Fri Sep 23 01:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:29.842464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336ba2-4966-4c84-abdf-33abd3ec1e33 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:29.842830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336ba2-4966-4c84-abdf-33abd3ec1e33 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:29.843172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336ba2-4966-4c84-abdf-33abd3ec1e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:29.933457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c4336ba2-4966-4c84-abdf-33abd3ec1e33 None None] MFA Rules not processed for user `c5a9f3eca3e34fffa7b218064e831d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:29.935209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c4336ba2-4966-4c84-abdf-33abd3ec1e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:30.011503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:30.092681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 705/1396] 10.222.0.64 () {58 vars in 1040 bytes} [Fri Sep 23 01:22:29 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:30.107854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35827a6-4894-41eb-bb9d-8907895de8ef None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:30.109537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f35827a6-4894-41eb-bb9d-8907895de8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:30.278745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35827a6-4894-41eb-bb9d-8907895de8ef None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:30.360242 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:30.434971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efcf860-6710-42c8-8ab1-89d6f92f1192 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:30.435618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcf860-6710-42c8-8ab1-89d6f92f1192 service swift] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:30.435906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcf860-6710-42c8-8ab1-89d6f92f1192 service swift] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:30.436245 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efcf860-6710-42c8-8ab1-89d6f92f1192 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:30.437987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3efcf860-6710-42c8-8ab1-89d6f92f1192 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:30.547580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcf860-6710-42c8-8ab1-89d6f92f1192 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:30.549243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efcf860-6710-42c8-8ab1-89d6f92f1192 service swift] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:30.561152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 692/1397] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:22:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:22:31.730799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fada2ad-e1fe-45c9-8a35-db395cdcf01d None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:31.731661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fada2ad-e1fe-45c9-8a35-db395cdcf01d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:31.731661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fada2ad-e1fe-45c9-8a35-db395cdcf01d None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:31.735229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 706/1398] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:31.739477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0ff55-b25f-4f20-aba0-8d9b93bfbac4 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:31.739477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0ff55-b25f-4f20-aba0-8d9b93bfbac4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:31.739477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0ff55-b25f-4f20-aba0-8d9b93bfbac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:31.830807 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d7e0ff55-b25f-4f20-aba0-8d9b93bfbac4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:31.832267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d7e0ff55-b25f-4f20-aba0-8d9b93bfbac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:31.905146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:31.986268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 693/1399] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:32.283425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fada2ad-e1fe-45c9-8a35-db395cdcf01d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:32.286085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7fada2ad-e1fe-45c9-8a35-db395cdcf01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:32.517648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fada2ad-e1fe-45c9-8a35-db395cdcf01d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:32.596183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:32.692385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af597c-5812-4c28-8d14-2954dac56567 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:32.693016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af597c-5812-4c28-8d14-2954dac56567 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:32.693314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af597c-5812-4c28-8d14-2954dac56567 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:32.693578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af597c-5812-4c28-8d14-2954dac56567 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:32.695359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d3af597c-5812-4c28-8d14-2954dac56567 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:32.849222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af597c-5812-4c28-8d14-2954dac56567 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:32.851942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af597c-5812-4c28-8d14-2954dac56567 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:32.906296 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 707/1400] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:35.343127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e038443-0fc7-4843-bd51-4fe7091b9f67 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:35.344367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e038443-0fc7-4843-bd51-4fe7091b9f67 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:35.344740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e038443-0fc7-4843-bd51-4fe7091b9f67 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:35.346391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 694/1401] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:35.352516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9346e-a890-4b5f-ad7f-4526111597c5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:35.353142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9346e-a890-4b5f-ad7f-4526111597c5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:35.353142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9346e-a890-4b5f-ad7f-4526111597c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:35.471612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:35.481503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-96f9346e-a890-4b5f-ad7f-4526111597c5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:35.483171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-96f9346e-a890-4b5f-ad7f-4526111597c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:35.658061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 708/1402] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:36.004811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e038443-0fc7-4843-bd51-4fe7091b9f67 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:36.007129 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0e038443-0fc7-4843-bd51-4fe7091b9f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:36.283522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e038443-0fc7-4843-bd51-4fe7091b9f67 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:36.335049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:36.498223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb971716-50ee-4f7d-a95f-a150194d57f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:36.499950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb971716-50ee-4f7d-a95f-a150194d57f6 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:36.500243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb971716-50ee-4f7d-a95f-a150194d57f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:36.500517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb971716-50ee-4f7d-a95f-a150194d57f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:36.502393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-eb971716-50ee-4f7d-a95f-a150194d57f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:36.643832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb971716-50ee-4f7d-a95f-a150194d57f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:36.645647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb971716-50ee-4f7d-a95f-a150194d57f6 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:36.673032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 695/1403] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:22:39.325451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e4dc9-3a33-4779-b2fe-0dcf80650b61 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:39.326080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e4dc9-3a33-4779-b2fe-0dcf80650b61 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:39.326080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e4dc9-3a33-4779-b2fe-0dcf80650b61 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:39.327395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 709/1404] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:39.333204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b11b-7c89-47f8-b352-56c3df32281e None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:39.333686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b11b-7c89-47f8-b352-56c3df32281e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:39.333771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b11b-7c89-47f8-b352-56c3df32281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:39.383281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:39.410613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-5c09b11b-7c89-47f8-b352-56c3df32281e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:39.412007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5c09b11b-7c89-47f8-b352-56c3df32281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:39.529612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 696/1405] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:22:39.828611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e4dc9-3a33-4779-b2fe-0dcf80650b61 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:39.830159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9e1e4dc9-3a33-4779-b2fe-0dcf80650b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:39.999284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:40.039682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e4dc9-3a33-4779-b2fe-0dcf80650b61 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:40.223155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1203d8-bc96-403c-aeba-6827c25d17ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:40.223804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1203d8-bc96-403c-aeba-6827c25d17ff service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:40.224082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1203d8-bc96-403c-aeba-6827c25d17ff service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:40.224355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1203d8-bc96-403c-aeba-6827c25d17ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:40.226414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5f1203d8-bc96-403c-aeba-6827c25d17ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:40.348571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1203d8-bc96-403c-aeba-6827c25d17ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:40.354326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1203d8-bc96-403c-aeba-6827c25d17ff service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:40.354942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:40.379248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 710/1406] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:42.838302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914787b1-1be7-49bd-a808-27ca40b00ec1 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:42.839205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914787b1-1be7-49bd-a808-27ca40b00ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:42.839205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914787b1-1be7-49bd-a808-27ca40b00ec1 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:42.844163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 697/1407] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:22:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:22:42.846812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fa504-744c-444a-a045-01dccb4191df None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:42.850244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fa504-744c-444a-a045-01dccb4191df None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:42.850635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6fa504-744c-444a-a045-01dccb4191df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:42.930209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-4c6fa504-744c-444a-a045-01dccb4191df None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:42.931607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4c6fa504-744c-444a-a045-01dccb4191df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:43.078329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 711/1408] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:43.398221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914787b1-1be7-49bd-a808-27ca40b00ec1 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:22:43.400240 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-914787b1-1be7-49bd-a808-27ca40b00ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:43.540724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:43.593328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914787b1-1be7-49bd-a808-27ca40b00ec1 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:22:43.744533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9590ab-1da3-4685-b6a8-7f7e25c72bda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:22:43.745473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9590ab-1da3-4685-b6a8-7f7e25c72bda service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:43.745744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9590ab-1da3-4685-b6a8-7f7e25c72bda service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:43.746115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9590ab-1da3-4685-b6a8-7f7e25c72bda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:43.748438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2a9590ab-1da3-4685-b6a8-7f7e25c72bda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:43.861948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:43.888939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9590ab-1da3-4685-b6a8-7f7e25c72bda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:22:43.890611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9590ab-1da3-4685-b6a8-7f7e25c72bda service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:22:43.916561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 698/1409] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:22:53.076684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=70 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:53.913911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=12 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:56.449537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e50914-6adb-414e-b3d5-568aa71b67f7 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:56.450225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e50914-6adb-414e-b3d5-568aa71b67f7 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:56.450225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e50914-6adb-414e-b3d5-568aa71b67f7 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:56.451622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 712/1410] 10.222.0.64 () {54 vars in 918 bytes} [Fri Sep 23 01:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:56.457878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba23fa-6ad3-4b91-98bc-c7694f04e0c7 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:56.458722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba23fa-6ad3-4b91-98bc-c7694f04e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:56.458722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba23fa-6ad3-4b91-98bc-c7694f04e0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:56.542997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8eba23fa-6ad3-4b91-98bc-c7694f04e0c7 None None] MFA Rules not processed for user `9f73bca3896440258430cfce042faa42`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:56.544253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8eba23fa-6ad3-4b91-98bc-c7694f04e0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:56.659464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 699/1411] 10.222.0.64 () {58 vars in 1015 bytes} [Fri Sep 23 01:22:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:22:56.675246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd57e86-9fae-4678-b22f-9b2d6342e968 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:56.675534 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd57e86-9fae-4678-b22f-9b2d6342e968 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:56.675865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd57e86-9fae-4678-b22f-9b2d6342e968 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:56.677281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 713/1412] 10.222.0.64 () {54 vars in 918 bytes} [Fri Sep 23 01:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:56.683513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777ef2e-85cb-48aa-9a0e-21b10a70c69b None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:56.683819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777ef2e-85cb-48aa-9a0e-21b10a70c69b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:56.684094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777ef2e-85cb-48aa-9a0e-21b10a70c69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:56.768913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:56.771869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8777ef2e-85cb-48aa-9a0e-21b10a70c69b None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:56.773140 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8777ef2e-85cb-48aa-9a0e-21b10a70c69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:56.901719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 700/1413] 10.222.0.64 () {58 vars in 1015 bytes} [Fri Sep 23 01:22:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:22:59.560541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da57aa-c4bc-4dd1-8414-d8ad353c68bf None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:59.561116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da57aa-c4bc-4dd1-8414-d8ad353c68bf None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:59.561116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da57aa-c4bc-4dd1-8414-d8ad353c68bf None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:59.562507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:22:59.568489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4d8c-a3a3-41d5-b830-917141cecfcd None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:22:59.569026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4d8c-a3a3-41d5-b830-917141cecfcd None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:22:59.569126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e4d8c-a3a3-41d5-b830-917141cecfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:22:59.644208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-eb3e4d8c-a3a3-41d5-b830-917141cecfcd None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:22:59.646193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-eb3e4d8c-a3a3-41d5-b830-917141cecfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:22:59.714386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:22:59.757426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:00.049072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da57aa-c4bc-4dd1-8414-d8ad353c68bf None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:00.050620 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f9da57aa-c4bc-4dd1-8414-d8ad353c68bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:00.215334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da57aa-c4bc-4dd1-8414-d8ad353c68bf None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:00.294162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:00.338712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a379bf-38f8-4f05-a762-02b702e3ff15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:00.339340 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a379bf-38f8-4f05-a762-02b702e3ff15 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:00.339680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a379bf-38f8-4f05-a762-02b702e3ff15 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:00.339964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a379bf-38f8-4f05-a762-02b702e3ff15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:00.342075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-44a379bf-38f8-4f05-a762-02b702e3ff15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:00.447717 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a379bf-38f8-4f05-a762-02b702e3ff15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:00.451716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a379bf-38f8-4f05-a762-02b702e3ff15 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:00.471597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 715/1416] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:01.691856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de4794-aaab-4e43-a755-f6a84a46a5f6 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:01.693554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de4794-aaab-4e43-a755-f6a84a46a5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:01.693554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de4794-aaab-4e43-a755-f6a84a46a5f6 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:01.693788 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:01.700549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6db00-69ad-4daa-ad40-476a21f27191 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:01.701091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6db00-69ad-4daa-ad40-476a21f27191 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:01.701091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6db00-69ad-4daa-ad40-476a21f27191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:01.785695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-abe6db00-69ad-4daa-ad40-476a21f27191 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:01.786942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-abe6db00-69ad-4daa-ad40-476a21f27191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:01.819740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:01.907749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:02.064682 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de4794-aaab-4e43-a755-f6a84a46a5f6 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:02.066162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-19de4794-aaab-4e43-a755-f6a84a46a5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:02.220446 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de4794-aaab-4e43-a755-f6a84a46a5f6 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:02.265477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:02.348919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2aa56-f4a5-4e77-addb-37fd748a7f54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:02.349421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2aa56-f4a5-4e77-addb-37fd748a7f54 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:02.349700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2aa56-f4a5-4e77-addb-37fd748a7f54 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:02.349971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2aa56-f4a5-4e77-addb-37fd748a7f54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:02.351692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d3b2aa56-f4a5-4e77-addb-37fd748a7f54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:02.489434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2aa56-f4a5-4e77-addb-37fd748a7f54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:02.491106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2aa56-f4a5-4e77-addb-37fd748a7f54 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:02.512264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 703/1419] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:03.699143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743a8a8-1151-4b0e-b978-0e7b04c8b086 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:03.699765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743a8a8-1151-4b0e-b978-0e7b04c8b086 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:03.699765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743a8a8-1151-4b0e-b978-0e7b04c8b086 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:03.701142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:23:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:03.708106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a111c87-a714-4cd6-a554-20f1b88cd755 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:03.708641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a111c87-a714-4cd6-a554-20f1b88cd755 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:03.708641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a111c87-a714-4cd6-a554-20f1b88cd755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:03.768904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:03.793168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8a111c87-a714-4cd6-a554-20f1b88cd755 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:03.794438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8a111c87-a714-4cd6-a554-20f1b88cd755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:03.905772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:04.063287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743a8a8-1151-4b0e-b978-0e7b04c8b086 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:04.064837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2743a8a8-1151-4b0e-b978-0e7b04c8b086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:04.224212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743a8a8-1151-4b0e-b978-0e7b04c8b086 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:04.237221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:04.347465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50df8b-22bc-4f30-9b99-943687d5cb14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:04.348748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50df8b-22bc-4f30-9b99-943687d5cb14 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:04.348748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50df8b-22bc-4f30-9b99-943687d5cb14 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:04.348748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50df8b-22bc-4f30-9b99-943687d5cb14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:04.350315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-dc50df8b-22bc-4f30-9b99-943687d5cb14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:04.452029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50df8b-22bc-4f30-9b99-943687d5cb14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:04.453676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50df8b-22bc-4f30-9b99-943687d5cb14 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:04.474068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 718/1422] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:05.641078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dec08b-1518-40fd-9091-9057c4232df0 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:05.641688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dec08b-1518-40fd-9091-9057c4232df0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:05.641688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dec08b-1518-40fd-9091-9057c4232df0 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:05.643074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:05.650011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364d034-7fbc-4f40-9590-aa4509341fad None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:05.650597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364d034-7fbc-4f40-9590-aa4509341fad None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:05.650597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9364d034-7fbc-4f40-9590-aa4509341fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:05.673352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:05.736159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9364d034-7fbc-4f40-9590-aa4509341fad None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:05.737402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9364d034-7fbc-4f40-9590-aa4509341fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:05.858794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:06.142459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dec08b-1518-40fd-9091-9057c4232df0 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:06.144064 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a0dec08b-1518-40fd-9091-9057c4232df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:06.267910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:06.309917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dec08b-1518-40fd-9091-9057c4232df0 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:06.443482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a85308-7dbf-49f7-bbfe-d013b8a6f1af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:06.444130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a85308-7dbf-49f7-bbfe-d013b8a6f1af service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:06.444415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a85308-7dbf-49f7-bbfe-d013b8a6f1af service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:06.444696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a85308-7dbf-49f7-bbfe-d013b8a6f1af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:06.446333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-83a85308-7dbf-49f7-bbfe-d013b8a6f1af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:06.533644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:06.548882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a85308-7dbf-49f7-bbfe-d013b8a6f1af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:06.550540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a85308-7dbf-49f7-bbfe-d013b8a6f1af service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:06.569101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 706/1425] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:07.721470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40efe9-ccb0-40e4-9b20-8342847a6935 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:07.722063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40efe9-ccb0-40e4-9b20-8342847a6935 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:07.722063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40efe9-ccb0-40e4-9b20-8342847a6935 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:07.723418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:23:07.731586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b24da-54e9-4ed4-8eb3-9e650a9eb1fe None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:07.732197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b24da-54e9-4ed4-8eb3-9e650a9eb1fe None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:07.732197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b24da-54e9-4ed4-8eb3-9e650a9eb1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:07.816117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f96b24da-54e9-4ed4-8eb3-9e650a9eb1fe None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:07.817367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f96b24da-54e9-4ed4-8eb3-9e650a9eb1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:07.963137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:08.093143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40efe9-ccb0-40e4-9b20-8342847a6935 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:08.094654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6f40efe9-ccb0-40e4-9b20-8342847a6935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:08.199388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:08.265678 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40efe9-ccb0-40e4-9b20-8342847a6935 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:08.386513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2a94f-76fa-4dc1-8801-57fc30441a49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:08.387158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2a94f-76fa-4dc1-8801-57fc30441a49 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:08.387440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2a94f-76fa-4dc1-8801-57fc30441a49 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:08.387746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2a94f-76fa-4dc1-8801-57fc30441a49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:08.389499 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-96c2a94f-76fa-4dc1-8801-57fc30441a49 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:08.442344 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:08.486294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2a94f-76fa-4dc1-8801-57fc30441a49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:08.488129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2a94f-76fa-4dc1-8801-57fc30441a49 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:08.506214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 721/1428] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:09.673998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f90efd-a2f7-4687-80d9-8e597d11b75b None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:09.674580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f90efd-a2f7-4687-80d9-8e597d11b75b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:09.674580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f90efd-a2f7-4687-80d9-8e597d11b75b None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:09.676017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:09.683067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c2338-6a74-466a-bbfa-00c649343852 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:09.683680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c2338-6a74-466a-bbfa-00c649343852 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:09.683782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c2338-6a74-466a-bbfa-00c649343852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:09.768573 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a07c2338-6a74-466a-bbfa-00c649343852 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:09.769863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a07c2338-6a74-466a-bbfa-00c649343852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:09.994703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:10.150701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f90efd-a2f7-4687-80d9-8e597d11b75b None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:10.153060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-89f90efd-a2f7-4687-80d9-8e597d11b75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:10.221704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:10.338253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f90efd-a2f7-4687-80d9-8e597d11b75b None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:10.465515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c18323-7be0-4c0d-99ca-f13fcd2ed65a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:10.466128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18323-7be0-4c0d-99ca-f13fcd2ed65a service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:10.466400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18323-7be0-4c0d-99ca-f13fcd2ed65a service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:10.466669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18323-7be0-4c0d-99ca-f13fcd2ed65a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:10.468324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-27c18323-7be0-4c0d-99ca-f13fcd2ed65a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:10.475511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:10.565507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c18323-7be0-4c0d-99ca-f13fcd2ed65a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:10.567206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c18323-7be0-4c0d-99ca-f13fcd2ed65a service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:10.586514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 709/1431] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:11.750422 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1119cf3-dbee-496a-b14e-c7fbe1b813bf None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:11.751027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1119cf3-dbee-496a-b14e-c7fbe1b813bf None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:11.751027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1119cf3-dbee-496a-b14e-c7fbe1b813bf None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:11.752568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:11.759600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9f5cb-a223-4b4e-8bda-0218ec37eefb None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:11.760145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9f5cb-a223-4b4e-8bda-0218ec37eefb None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:11.760145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9f5cb-a223-4b4e-8bda-0218ec37eefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:11.846841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-82a9f5cb-a223-4b4e-8bda-0218ec37eefb None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:11.848148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-82a9f5cb-a223-4b4e-8bda-0218ec37eefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:11.920286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:11.954619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:23:12.105265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1119cf3-dbee-496a-b14e-c7fbe1b813bf None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:12.106715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d1119cf3-dbee-496a-b14e-c7fbe1b813bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:12.124986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:12.257230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1119cf3-dbee-496a-b14e-c7fbe1b813bf None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:12.354201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:12.386776 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ece06-f4e1-4365-a9e0-1cc51410d2b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:12.387394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ece06-f4e1-4365-a9e0-1cc51410d2b0 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:12.387716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ece06-f4e1-4365-a9e0-1cc51410d2b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:12.387994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ece06-f4e1-4365-a9e0-1cc51410d2b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:12.389628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-393ece06-f4e1-4365-a9e0-1cc51410d2b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:12.496487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ece06-f4e1-4365-a9e0-1cc51410d2b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:12.498077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ece06-f4e1-4365-a9e0-1cc51410d2b0 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:12.517178 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 724/1434] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:13.708784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d93378-0f0a-4d62-9dc2-e39b359e2b9e None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:13.709382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d93378-0f0a-4d62-9dc2-e39b359e2b9e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:13.709382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d93378-0f0a-4d62-9dc2-e39b359e2b9e None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:13.710942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:13.718292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017fce3-f510-48ee-aae3-4eea6c70c8bc None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:13.718834 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017fce3-f510-48ee-aae3-4eea6c70c8bc None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:13.718948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017fce3-f510-48ee-aae3-4eea6c70c8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:13.802754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9017fce3-f510-48ee-aae3-4eea6c70c8bc None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:13.804037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9017fce3-f510-48ee-aae3-4eea6c70c8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:13.853937 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:13.916066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:14.079492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d93378-0f0a-4d62-9dc2-e39b359e2b9e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:14.081266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-87d93378-0f0a-4d62-9dc2-e39b359e2b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:14.245979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d93378-0f0a-4d62-9dc2-e39b359e2b9e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:14.314108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:14.410401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3f854-9100-48cc-8b33-70cbf459a847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:14.411681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3f854-9100-48cc-8b33-70cbf459a847 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:14.411967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3f854-9100-48cc-8b33-70cbf459a847 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:14.412245 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3f854-9100-48cc-8b33-70cbf459a847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:14.414232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cab3f854-9100-48cc-8b33-70cbf459a847 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:14.523369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3f854-9100-48cc-8b33-70cbf459a847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:14.525071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3f854-9100-48cc-8b33-70cbf459a847 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:14.547079 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 712/1437] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:15.775731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdd39f-bb00-418b-b6ba-33d50598181b None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:15.776808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdd39f-bb00-418b-b6ba-33d50598181b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:15.776808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cdd39f-bb00-418b-b6ba-33d50598181b None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:15.778161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:23:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:15.784853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288aaf72-d4d6-4385-af58-91267a09c3a2 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:15.791610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288aaf72-d4d6-4385-af58-91267a09c3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:15.791610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288aaf72-d4d6-4385-af58-91267a09c3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:15.875978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-288aaf72-d4d6-4385-af58-91267a09c3a2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:15.877374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-288aaf72-d4d6-4385-af58-91267a09c3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:15.879601 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:15.998486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:16.156469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdd39f-bb00-418b-b6ba-33d50598181b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:16.158023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-38cdd39f-bb00-418b-b6ba-33d50598181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:16.322814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cdd39f-bb00-418b-b6ba-33d50598181b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:16.355388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:16.449967 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d65ca-ea5c-434f-ba34-35a5c7d56710 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:16.450586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d65ca-ea5c-434f-ba34-35a5c7d56710 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:16.450956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d65ca-ea5c-434f-ba34-35a5c7d56710 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:16.451231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d65ca-ea5c-434f-ba34-35a5c7d56710 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:16.453009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-595d65ca-ea5c-434f-ba34-35a5c7d56710 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:16.549955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d65ca-ea5c-434f-ba34-35a5c7d56710 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:16.551580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d65ca-ea5c-434f-ba34-35a5c7d56710 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:16.572058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 727/1440] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:18.705166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1e000-f858-4a9e-b25e-f6a192137eb1 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:18.706136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1e000-f858-4a9e-b25e-f6a192137eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:18.706136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1e000-f858-4a9e-b25e-f6a192137eb1 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:18.707312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:18.714286 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa2bda-a81e-4649-a6c5-2f416e4b230c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:18.714768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa2bda-a81e-4649-a6c5-2f416e4b230c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:18.714881 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa2bda-a81e-4649-a6c5-2f416e4b230c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:18.758439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:18.785016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-34fa2bda-a81e-4649-a6c5-2f416e4b230c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:18.786287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-34fa2bda-a81e-4649-a6c5-2f416e4b230c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:18.900406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:19.187546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1e000-f858-4a9e-b25e-f6a192137eb1 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:19.189306 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0fd1e000-f858-4a9e-b25e-f6a192137eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:19.304482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1e000-f858-4a9e-b25e-f6a192137eb1 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:19.338852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:19.382689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df434c81-4700-4002-8fd9-aab32bd9e3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:19.383318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df434c81-4700-4002-8fd9-aab32bd9e3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:19.383625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df434c81-4700-4002-8fd9-aab32bd9e3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:19.383904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df434c81-4700-4002-8fd9-aab32bd9e3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:19.386335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-df434c81-4700-4002-8fd9-aab32bd9e3f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:19.524946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df434c81-4700-4002-8fd9-aab32bd9e3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:19.526315 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df434c81-4700-4002-8fd9-aab32bd9e3f5 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:19.546063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 715/1443] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:21.943075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fba391-6d04-47e9-b15b-f06fe5933aad None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:21.943892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fba391-6d04-47e9-b15b-f06fe5933aad None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:21.943892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fba391-6d04-47e9-b15b-f06fe5933aad None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:21.945537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:21.951410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb19366-3749-45ff-9142-2b40139d118b None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:21.958417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb19366-3749-45ff-9142-2b40139d118b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:21.958417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb19366-3749-45ff-9142-2b40139d118b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:22.026648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-adb19366-3749-45ff-9142-2b40139d118b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:22.027930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-adb19366-3749-45ff-9142-2b40139d118b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:22.042702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:22.141634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:23:22.431944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fba391-6d04-47e9-b15b-f06fe5933aad None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:22.433662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-44fba391-6d04-47e9-b15b-f06fe5933aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:22.546668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fba391-6d04-47e9-b15b-f06fe5933aad None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:22.560005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:22.623506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a30c4-ecc1-4a9f-b74f-c1b6803747fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:22.624149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a30c4-ecc1-4a9f-b74f-c1b6803747fe service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:22.624425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a30c4-ecc1-4a9f-b74f-c1b6803747fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:22.624693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a30c4-ecc1-4a9f-b74f-c1b6803747fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:22.626276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e49a30c4-ecc1-4a9f-b74f-c1b6803747fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:22.746701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a30c4-ecc1-4a9f-b74f-c1b6803747fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:22.748170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a30c4-ecc1-4a9f-b74f-c1b6803747fe service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:22.768346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 730/1446] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:25.450771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e9eab-3d0d-4354-9af0-347e1ee89976 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:25.451364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e9eab-3d0d-4354-9af0-347e1ee89976 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:25.451364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e9eab-3d0d-4354-9af0-347e1ee89976 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:25.452806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:23:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:23:25.460209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8cafe-b69a-4ace-9981-385baa97af7c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:25.460960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8cafe-b69a-4ace-9981-385baa97af7c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:25.461127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8cafe-b69a-4ace-9981-385baa97af7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:25.517504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:25.540812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-7fb8cafe-b69a-4ace-9981-385baa97af7c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:25.542687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7fb8cafe-b69a-4ace-9981-385baa97af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:25.649972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:23:25.934744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e9eab-3d0d-4354-9af0-347e1ee89976 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:25.936327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d05e9eab-3d0d-4354-9af0-347e1ee89976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:26.183078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e9eab-3d0d-4354-9af0-347e1ee89976 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:26.232738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:26.268748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b00785-4511-43e3-aea0-13e157c2f548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:26.269356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b00785-4511-43e3-aea0-13e157c2f548 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:26.269631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b00785-4511-43e3-aea0-13e157c2f548 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:26.269900 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b00785-4511-43e3-aea0-13e157c2f548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:26.271576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-11b00785-4511-43e3-aea0-13e157c2f548 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:26.387929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b00785-4511-43e3-aea0-13e157c2f548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:26.389177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b00785-4511-43e3-aea0-13e157c2f548 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:26.408473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 718/1449] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:28.792114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c84b8f-d332-40d8-a193-02d73745260d None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:28.792114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c84b8f-d332-40d8-a193-02d73745260d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:28.792114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c84b8f-d332-40d8-a193-02d73745260d None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:28.792114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:28.794544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dada-042e-4878-9ede-7f0b95de59f2 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:28.794951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dada-042e-4878-9ede-7f0b95de59f2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:28.795167 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74dada-042e-4878-9ede-7f0b95de59f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:28.875956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-fc74dada-042e-4878-9ede-7f0b95de59f2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:28.877220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fc74dada-042e-4878-9ede-7f0b95de59f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:28.903507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:28.993737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:29.281432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c84b8f-d332-40d8-a193-02d73745260d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:29.282979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c1c84b8f-d332-40d8-a193-02d73745260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:29.395997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c84b8f-d332-40d8-a193-02d73745260d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:29.417737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:29.475422 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d95954-e040-4a23-a6f1-dab7fda93824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:29.476072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d95954-e040-4a23-a6f1-dab7fda93824 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:29.476353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d95954-e040-4a23-a6f1-dab7fda93824 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:29.476628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d95954-e040-4a23-a6f1-dab7fda93824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:29.478240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-23d95954-e040-4a23-a6f1-dab7fda93824 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:29.611852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d95954-e040-4a23-a6f1-dab7fda93824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:29.613255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d95954-e040-4a23-a6f1-dab7fda93824 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:29.632901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 733/1452] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:32.091578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9c90-3c04-4a5a-ae3e-4b7d1f83a339 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:32.092200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9c90-3c04-4a5a-ae3e-4b7d1f83a339 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:32.092200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a9c90-3c04-4a5a-ae3e-4b7d1f83a339 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:32.093573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:32.100024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ae75c-87db-4696-b381-f1680641ba1f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:32.100655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ae75c-87db-4696-b381-f1680641ba1f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:32.100785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ae75c-87db-4696-b381-f1680641ba1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:32.186984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:32.190964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-561ae75c-87db-4696-b381-f1680641ba1f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:32.192400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-561ae75c-87db-4696-b381-f1680641ba1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:32.298801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:32.586259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a9c90-3c04-4a5a-ae3e-4b7d1f83a339 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:32.587854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-934a9c90-3c04-4a5a-ae3e-4b7d1f83a339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:32.695915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a9c90-3c04-4a5a-ae3e-4b7d1f83a339 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:32.745883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:32.772208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f3569-e3d0-43a2-933f-10d36f606a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:32.772917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f3569-e3d0-43a2-933f-10d36f606a3f service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:32.773206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f3569-e3d0-43a2-933f-10d36f606a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:32.773444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f3569-e3d0-43a2-933f-10d36f606a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:32.775084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e21f3569-e3d0-43a2-933f-10d36f606a3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:32.890884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f3569-e3d0-43a2-933f-10d36f606a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:32.892371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f3569-e3d0-43a2-933f-10d36f606a3f service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:32.911390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 721/1455] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:35.181257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5d9e1-85a9-4e9c-b3c9-3a362dc4cc4c None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:35.182358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5d9e1-85a9-4e9c-b3c9-3a362dc4cc4c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:35.182358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5d9e1-85a9-4e9c-b3c9-3a362dc4cc4c None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:35.184403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:23:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:35.190733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef59b4d-2235-4d47-9b49-3014cd10af4b None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:35.191324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef59b4d-2235-4d47-9b49-3014cd10af4b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:35.191324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef59b4d-2235-4d47-9b49-3014cd10af4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:35.268896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-7ef59b4d-2235-4d47-9b49-3014cd10af4b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:35.270143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7ef59b4d-2235-4d47-9b49-3014cd10af4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:35.302270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:35.373637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:35.664173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5d9e1-85a9-4e9c-b3c9-3a362dc4cc4c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:35.665768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a8f5d9e1-85a9-4e9c-b3c9-3a362dc4cc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:35.775385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5d9e1-85a9-4e9c-b3c9-3a362dc4cc4c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:35.805855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:35.855159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4106010-2665-4b82-b63a-2d022f474e04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:35.855829 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4106010-2665-4b82-b63a-2d022f474e04 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:35.856115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4106010-2665-4b82-b63a-2d022f474e04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:35.856392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4106010-2665-4b82-b63a-2d022f474e04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:35.858077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c4106010-2665-4b82-b63a-2d022f474e04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:35.985412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4106010-2665-4b82-b63a-2d022f474e04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:35.986861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4106010-2665-4b82-b63a-2d022f474e04 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:36.009342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 736/1458] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:38.334909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e1e5-72e0-4bbc-94de-84739e1b08e5 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:38.335509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e1e5-72e0-4bbc-94de-84739e1b08e5 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:38.335509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e1e5-72e0-4bbc-94de-84739e1b08e5 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:38.336869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:38.342716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2d48-d9c4-4551-9a14-bf05117fd0ed None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:38.343240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2d48-d9c4-4551-9a14-bf05117fd0ed None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:38.343240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2d48-d9c4-4551-9a14-bf05117fd0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:38.419728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3daf2d48-d9c4-4551-9a14-bf05117fd0ed None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:38.420994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3daf2d48-d9c4-4551-9a14-bf05117fd0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:38.430709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:38.531634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:38.821331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4e1e5-72e0-4bbc-94de-84739e1b08e5 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:38.823482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c5a4e1e5-72e0-4bbc-94de-84739e1b08e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:38.962221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4e1e5-72e0-4bbc-94de-84739e1b08e5 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:39.026713 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:39.047272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054f50c-649c-410e-acf5-3ec847a6b4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:39.048382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054f50c-649c-410e-acf5-3ec847a6b4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:39.048382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054f50c-649c-410e-acf5-3ec847a6b4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:39.048620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054f50c-649c-410e-acf5-3ec847a6b4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:39.050107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9054f50c-649c-410e-acf5-3ec847a6b4f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:39.169774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054f50c-649c-410e-acf5-3ec847a6b4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:39.171313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054f50c-649c-410e-acf5-3ec847a6b4f3 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:39.190891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 724/1461] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:23:41.465819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b6121-d8cc-40bc-92af-c848ab4f6c4f None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:41.466393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b6121-d8cc-40bc-92af-c848ab4f6c4f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:41.466393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b6121-d8cc-40bc-92af-c848ab4f6c4f None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:41.467809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:41.473407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1fcd1-d0ce-4174-bf6e-2f492337c661 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:41.473922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1fcd1-d0ce-4174-bf6e-2f492337c661 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:41.474034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1fcd1-d0ce-4174-bf6e-2f492337c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:41.544363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-dbd1fcd1-d0ce-4174-bf6e-2f492337c661 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:41.545611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-dbd1fcd1-d0ce-4174-bf6e-2f492337c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:41.586469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:41.650700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:41.934620 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b6121-d8cc-40bc-92af-c848ab4f6c4f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:41.936245 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-949b6121-d8cc-40bc-92af-c848ab4f6c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:42.086006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b6121-d8cc-40bc-92af-c848ab4f6c4f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:42.153489 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:42.204973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcea6f7-4d13-4bd9-b7db-2672d4f1e679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:42.204973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcea6f7-4d13-4bd9-b7db-2672d4f1e679 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:42.205326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcea6f7-4d13-4bd9-b7db-2672d4f1e679 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:42.205419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcea6f7-4d13-4bd9-b7db-2672d4f1e679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:42.207265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2dcea6f7-4d13-4bd9-b7db-2672d4f1e679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:42.327352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcea6f7-4d13-4bd9-b7db-2672d4f1e679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:42.329040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcea6f7-4d13-4bd9-b7db-2672d4f1e679 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:42.350014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 739/1464] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:44.629565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29084b7-d1fe-4b03-a7d3-042a4906ea43 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:44.630422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29084b7-d1fe-4b03-a7d3-042a4906ea43 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:44.630422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29084b7-d1fe-4b03-a7d3-042a4906ea43 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:44.632039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:44.638806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8b291-3827-476d-bbc9-7bc76d398d08 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:44.638806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8b291-3827-476d-bbc9-7bc76d398d08 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:44.639679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8b291-3827-476d-bbc9-7bc76d398d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:44.711379 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-05d8b291-3827-476d-bbc9-7bc76d398d08 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:44.712557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-05d8b291-3827-476d-bbc9-7bc76d398d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:44.733976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:44.819393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:45.103885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29084b7-d1fe-4b03-a7d3-042a4906ea43 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:45.105451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e29084b7-d1fe-4b03-a7d3-042a4906ea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:45.218788 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29084b7-d1fe-4b03-a7d3-042a4906ea43 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:45.295016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:45.301565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a5abf-3a15-47a7-b550-6c8142ea9ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:45.302118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a5abf-3a15-47a7-b550-6c8142ea9ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:45.302430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a5abf-3a15-47a7-b550-6c8142ea9ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:45.302731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a5abf-3a15-47a7-b550-6c8142ea9ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:45.304382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-813a5abf-3a15-47a7-b550-6c8142ea9ec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:45.432540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a5abf-3a15-47a7-b550-6c8142ea9ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:45.434116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a5abf-3a15-47a7-b550-6c8142ea9ec3 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:45.460257 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 727/1467] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:47.682565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e140f-779b-4de9-854a-5069ede14ec3 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:47.683242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e140f-779b-4de9-854a-5069ede14ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:47.683242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0e140f-779b-4de9-854a-5069ede14ec3 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:47.684555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:23:47.690522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dae9fe-d336-4900-9caa-3f63ecc3aa73 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:47.691153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dae9fe-d336-4900-9caa-3f63ecc3aa73 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:47.691153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dae9fe-d336-4900-9caa-3f63ecc3aa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:47.764994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b0dae9fe-d336-4900-9caa-3f63ecc3aa73 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:23:47.766234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b0dae9fe-d336-4900-9caa-3f63ecc3aa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:47.826404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:47.890521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:23:48.178326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e140f-779b-4de9-854a-5069ede14ec3 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:23:48.179990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1f0e140f-779b-4de9-854a-5069ede14ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:48.307052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0e140f-779b-4de9-854a-5069ede14ec3 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:23:48.360704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:48.389082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63d902-e476-4b35-8db8-978dc136c1e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:23:48.389703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d902-e476-4b35-8db8-978dc136c1e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:23:48.389988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d902-e476-4b35-8db8-978dc136c1e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:23:48.390275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63d902-e476-4b35-8db8-978dc136c1e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:23:48.392719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1e63d902-e476-4b35-8db8-978dc136c1e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:23:48.505293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63d902-e476-4b35-8db8-978dc136c1e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:23:48.506732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63d902-e476-4b35-8db8-978dc136c1e2 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:23:48.526026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 742/1470] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:23:57.888913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=22 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:23:58.523130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=66 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:09.196052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690e195-04c6-41ca-a98f-01e74baec05e None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:09.196701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690e195-04c6-41ca-a98f-01e74baec05e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:09.196701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690e195-04c6-41ca-a98f-01e74baec05e None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:09.198365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 729/1471] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:24:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:24:09.204956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69bd1a-a52a-4c9e-9cfa-14e5c01f39fd None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:09.205517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69bd1a-a52a-4c9e-9cfa-14e5c01f39fd None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:09.205517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69bd1a-a52a-4c9e-9cfa-14e5c01f39fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:09.278954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3b69bd1a-a52a-4c9e-9cfa-14e5c01f39fd None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:24:09.280209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3b69bd1a-a52a-4c9e-9cfa-14e5c01f39fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:09.385494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 743/1472] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:24:09.695643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb1013-14ae-497b-b6ae-fd12ce61b689 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:09.695937 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb1013-14ae-497b-b6ae-fd12ce61b689 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:09.696212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb1013-14ae-497b-b6ae-fd12ce61b689 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:09.697605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 730/1473] 10.222.0.64 () {54 vars in 962 bytes} [Fri Sep 23 01:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:24:09.702783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d4b0b-155d-4960-a84f-e8d5cd258514 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:09.703076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d4b0b-155d-4960-a84f-e8d5cd258514 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:09.703356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d4b0b-155d-4960-a84f-e8d5cd258514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:09.784046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d06d4b0b-155d-4960-a84f-e8d5cd258514 None None] MFA Rules not processed for user `6f10c258fe5f45ccab1669fa32ed6029`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:24:09.785316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d06d4b0b-155d-4960-a84f-e8d5cd258514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:09.807248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:09.864201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 744/1474] 10.222.0.64 () {58 vars in 1059 bytes} [Fri Sep 23 01:24:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:24:09.875938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb1013-14ae-497b-b6ae-fd12ce61b689 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:24:09.877515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b9eb1013-14ae-497b-b6ae-fd12ce61b689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:09.995354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb1013-14ae-497b-b6ae-fd12ce61b689 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:24:10.075171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b9684-5d89-4c05-82dc-aef0bb37a19a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f10c258fe5f45ccab1669fa32ed6029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:24:10.075807 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b9684-5d89-4c05-82dc-aef0bb37a19a service glance] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:10.076083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b9684-5d89-4c05-82dc-aef0bb37a19a service glance] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:10.076383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b9684-5d89-4c05-82dc-aef0bb37a19a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:10.077970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-651b9684-5d89-4c05-82dc-aef0bb37a19a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:10.136307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:10.203217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b9684-5d89-4c05-82dc-aef0bb37a19a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:24:10.204933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b9684-5d89-4c05-82dc-aef0bb37a19a service glance] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:24:10.227648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 731/1475] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:24:12.590430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1e8b3-1f10-4165-a664-7a85369a5180 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:12.591164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1e8b3-1f10-4165-a664-7a85369a5180 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:12.591164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf1e8b3-1f10-4165-a664-7a85369a5180 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:12.592421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 745/1476] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:24:12.599116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8385-d657-4817-864d-c2929772418e None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:12.599833 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8385-d657-4817-864d-c2929772418e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:12.599833 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea8385-d657-4817-864d-c2929772418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:12.677978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-63ea8385-d657-4817-864d-c2929772418e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:24:12.679234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-63ea8385-d657-4817-864d-c2929772418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:12.787307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 732/1477] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:24:13.065864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1e8b3-1f10-4165-a664-7a85369a5180 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:24:13.067852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-baf1e8b3-1f10-4165-a664-7a85369a5180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:13.239323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf1e8b3-1f10-4165-a664-7a85369a5180 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:24:13.309475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:13.373822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d56c5-af95-43b8-b14e-b103c857a0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:24:13.374446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d56c5-af95-43b8-b14e-b103c857a0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:13.374711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d56c5-af95-43b8-b14e-b103c857a0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:13.374995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d56c5-af95-43b8-b14e-b103c857a0f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:13.375701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d56c5-af95-43b8-b14e-b103c857a0f5 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:24:13.377582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WARNING py.warnings [None req-eb2d56c5-af95-43b8-b14e-b103c857a0f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:24:13.377582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: warnings.warn(msg) Sep 23 01:24:13.377582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: Sep 23 01:24:13.378854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d56c5-af95-43b8-b14e-b103c857a0f5 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:24:13.403812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 746/1478] 10.222.0.64 () {56 vars in 1108 bytes} [Fri Sep 23 01:24:13 2022] GET /identity/v3/projects => generated 2553 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:24:15.673471 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515933c-307e-4af9-b074-1d1c7ad4e39f None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:15.674056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515933c-307e-4af9-b074-1d1c7ad4e39f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:15.674179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515933c-307e-4af9-b074-1d1c7ad4e39f None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:15.675935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 733/1479] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:24:15.682213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ab931-2e14-4a15-b0e9-86ebae5fc5d1 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:15.682738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ab931-2e14-4a15-b0e9-86ebae5fc5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:15.682876 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ab931-2e14-4a15-b0e9-86ebae5fc5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:15.757375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-246ab931-2e14-4a15-b0e9-86ebae5fc5d1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:24:15.759276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-246ab931-2e14-4a15-b0e9-86ebae5fc5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:15.867474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 747/1480] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:24:16.162758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515933c-307e-4af9-b074-1d1c7ad4e39f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:24:16.164339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9515933c-307e-4af9-b074-1d1c7ad4e39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:16.174458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:16.328689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515933c-307e-4af9-b074-1d1c7ad4e39f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:24:16.420540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:16.462252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8da48-7136-4c65-abe4-3052335909b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:24:16.463466 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8da48-7136-4c65-abe4-3052335909b3 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:16.463816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8da48-7136-4c65-abe4-3052335909b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:16.464130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8da48-7136-4c65-abe4-3052335909b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:16.465901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6fa8da48-7136-4c65-abe4-3052335909b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:16.571218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8da48-7136-4c65-abe4-3052335909b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:24:16.572906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8da48-7136-4c65-abe4-3052335909b3 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:24:16.593020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 734/1481] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:24:19.008213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d15ed-80c3-459a-874a-ecda71978229 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:19.008855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d15ed-80c3-459a-874a-ecda71978229 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:19.008855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d15ed-80c3-459a-874a-ecda71978229 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:19.010733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 748/1482] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:24:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:24:19.016630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf5d35-2264-4c57-a4c0-39d51fd47335 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:19.017162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf5d35-2264-4c57-a4c0-39d51fd47335 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:19.017256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf5d35-2264-4c57-a4c0-39d51fd47335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:19.091146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f0bf5d35-2264-4c57-a4c0-39d51fd47335 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:24:19.092320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f0bf5d35-2264-4c57-a4c0-39d51fd47335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:19.140942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:19.192638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 735/1483] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:24:19.511736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d15ed-80c3-459a-874a-ecda71978229 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:24:19.513279 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-938d15ed-80c3-459a-874a-ecda71978229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:19.538032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:19.630303 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d15ed-80c3-459a-874a-ecda71978229 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:24:19.707764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc59868-30b3-45a3-a9f0-1becfb3baba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:24:19.708386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc59868-30b3-45a3-a9f0-1becfb3baba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:19.708665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc59868-30b3-45a3-a9f0-1becfb3baba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:19.708950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc59868-30b3-45a3-a9f0-1becfb3baba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:19.710595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2fc59868-30b3-45a3-a9f0-1becfb3baba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:19.791960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:19.828776 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc59868-30b3-45a3-a9f0-1becfb3baba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:24:19.830284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc59868-30b3-45a3-a9f0-1becfb3baba3 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:24:19.850143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 749/1484] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:24:22.112386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336673a7-8b1d-453e-a3b4-ced93b46055d None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:22.113209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336673a7-8b1d-453e-a3b4-ced93b46055d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:22.113209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336673a7-8b1d-453e-a3b4-ced93b46055d None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:22.114899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 736/1485] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:24:22.121185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcfabe-9ac4-4198-b7e1-47fe82a34043 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:22.121720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcfabe-9ac4-4198-b7e1-47fe82a34043 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:22.121720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcfabe-9ac4-4198-b7e1-47fe82a34043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:22.201834 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-75bcfabe-9ac4-4198-b7e1-47fe82a34043 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:24:22.203420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-75bcfabe-9ac4-4198-b7e1-47fe82a34043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:22.315846 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 750/1486] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:24:22.604441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336673a7-8b1d-453e-a3b4-ced93b46055d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:24:22.605988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-336673a7-8b1d-453e-a3b4-ced93b46055d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:22.720333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336673a7-8b1d-453e-a3b4-ced93b46055d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:24:22.798263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:22.799790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30ee53-0fad-49a6-af44-f2b058689b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:24:22.800407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30ee53-0fad-49a6-af44-f2b058689b3a service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:22.800691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30ee53-0fad-49a6-af44-f2b058689b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:22.801010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30ee53-0fad-49a6-af44-f2b058689b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:22.802636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fd30ee53-0fad-49a6-af44-f2b058689b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:22.918752 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30ee53-0fad-49a6-af44-f2b058689b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:24:22.920255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30ee53-0fad-49a6-af44-f2b058689b3a service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:24:22.938837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 737/1487] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:24:25.263853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2aa4d1-2136-4dd9-ae83-1a7aa888afc6 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:25.264416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2aa4d1-2136-4dd9-ae83-1a7aa888afc6 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:25.264607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2aa4d1-2136-4dd9-ae83-1a7aa888afc6 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:25.266207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 751/1488] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:24:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:24:25.272386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b196f-5121-4709-aded-283b0fb1515d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:25.272896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b196f-5121-4709-aded-283b0fb1515d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:25.273015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b196f-5121-4709-aded-283b0fb1515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:25.346695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-021b196f-5121-4709-aded-283b0fb1515d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:24:25.348021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-021b196f-5121-4709-aded-283b0fb1515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:25.409101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:25.458679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 738/1489] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:24:25.751575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2aa4d1-2136-4dd9-ae83-1a7aa888afc6 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:24:25.753170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2f2aa4d1-2136-4dd9-ae83-1a7aa888afc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:25.792917 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:25.865006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2aa4d1-2136-4dd9-ae83-1a7aa888afc6 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:24:25.941441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2a7fd-55be-4648-800c-d4b6f75bc3f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:24:25.942171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2a7fd-55be-4648-800c-d4b6f75bc3f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:25.942459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2a7fd-55be-4648-800c-d4b6f75bc3f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:25.942733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2a7fd-55be-4648-800c-d4b6f75bc3f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:25.944465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5da2a7fd-55be-4648-800c-d4b6f75bc3f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:26.035649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:26.064874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2a7fd-55be-4648-800c-d4b6f75bc3f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:24:26.066331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2a7fd-55be-4648-800c-d4b6f75bc3f2 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:24:26.086406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 752/1490] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:24:28.844321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf075ad-37eb-4d00-98fa-ada2a2c752e2 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:28.844898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf075ad-37eb-4d00-98fa-ada2a2c752e2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:28.844898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf075ad-37eb-4d00-98fa-ada2a2c752e2 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:28.846272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 739/1491] 10.222.0.64 () {54 vars in 927 bytes} [Fri Sep 23 01:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:24:28.851857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85e640-d168-457b-b9e3-934294cb9ee9 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:28.853022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85e640-d168-457b-b9e3-934294cb9ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:28.853306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85e640-d168-457b-b9e3-934294cb9ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:28.928496 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ad85e640-d168-457b-b9e3-934294cb9ee9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:24:28.929774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ad85e640-d168-457b-b9e3-934294cb9ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:29.034190 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 753/1492] 10.222.0.64 () {58 vars in 1024 bytes} [Fri Sep 23 01:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:24:29.321476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf075ad-37eb-4d00-98fa-ada2a2c752e2 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:24:29.323043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9bf075ad-37eb-4d00-98fa-ada2a2c752e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:29.434590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf075ad-37eb-4d00-98fa-ada2a2c752e2 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:24:29.516266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb6bba-0826-4893-98f2-7b9e94189a73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:24:29.516879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb6bba-0826-4893-98f2-7b9e94189a73 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:24:29.517162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb6bba-0826-4893-98f2-7b9e94189a73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:24:29.517439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb6bba-0826-4893-98f2-7b9e94189a73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:24:29.519121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fefb6bba-0826-4893-98f2-7b9e94189a73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:24:29.528517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:29.634175 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb6bba-0826-4893-98f2-7b9e94189a73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:24:29.635675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb6bba-0826-4893-98f2-7b9e94189a73 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:24:29.654260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 740/1493] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:24:39.032645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=82 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:24:39.651938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=50 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:26:06.662277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca7b73-f6ba-4175-b0d4-2a25c3442c5d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:06.662841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca7b73-f6ba-4175-b0d4-2a25c3442c5d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:06.662950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca7b73-f6ba-4175-b0d4-2a25c3442c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:06.748567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f3ca7b73-f6ba-4175-b0d4-2a25c3442c5d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:26:06.749876 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f3ca7b73-f6ba-4175-b0d4-2a25c3442c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:26:06.863964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 754/1494] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:26:06.872506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7be8f6-6082-4c9d-87ab-b86acc209a80 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:06.873087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7be8f6-6082-4c9d-87ab-b86acc209a80 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:06.873087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7be8f6-6082-4c9d-87ab-b86acc209a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:06.894591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-0d7be8f6-6082-4c9d-87ab-b86acc209a80 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:26:06.898972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0d7be8f6-6082-4c9d-87ab-b86acc209a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:26:06.905329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 741/1495] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:26:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:26:06.911057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca7b73-f6ba-4175-b0d4-2a25c3442c5d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:26:06.922524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca7b73-f6ba-4175-b0d4-2a25c3442c5d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:26:06.927315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8cdf78-7562-4dcc-8d99-3d43c47e5154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:26:06.928058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cdf78-7562-4dcc-8d99-3d43c47e5154 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:06.928300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cdf78-7562-4dcc-8d99-3d43c47e5154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:06.928524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8cdf78-7562-4dcc-8d99-3d43c47e5154 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:06.929193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8cdf78-7562-4dcc-8d99-3d43c47e5154 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:26:06.929502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8cdf78-7562-4dcc-8d99-3d43c47e5154 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:26:06.931040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WARNING py.warnings [None req-2e8cdf78-7562-4dcc-8d99-3d43c47e5154 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:26:06.931040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: warnings.warn(msg) Sep 23 01:26:06.931040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: Sep 23 01:26:06.931976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8cdf78-7562-4dcc-8d99-3d43c47e5154 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:26:06.945370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 755/1496] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:26:06.954884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7be8f6-6082-4c9d-87ab-b86acc209a80 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:26:06.966841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7be8f6-6082-4c9d-87ab-b86acc209a80 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:26:06.973583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7a242-8365-4d5a-a221-970a36dc9be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:26:06.973583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7a242-8365-4d5a-a221-970a36dc9be4 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:06.973583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7a242-8365-4d5a-a221-970a36dc9be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:06.973583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7a242-8365-4d5a-a221-970a36dc9be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:06.974176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7a242-8365-4d5a-a221-970a36dc9be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:26:06.975484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: WARNING py.warnings [None req-e9a7a242-8365-4d5a-a221-970a36dc9be4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:26:06.975484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: warnings.warn(msg) Sep 23 01:26:06.975484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: Sep 23 01:26:06.976724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a7a242-8365-4d5a-a221-970a36dc9be4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:26:07.091644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 742/1497] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:26:06 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:26:07.096133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8cdf78-7562-4dcc-8d99-3d43c47e5154 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:26:07.107244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8cdf78-7562-4dcc-8d99-3d43c47e5154 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:26:07.160980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:26:07.170383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fe061-ccb1-49c9-a2d4-358c067c587b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:26:07.174426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fe061-ccb1-49c9-a2d4-358c067c587b admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:07.174426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fe061-ccb1-49c9-a2d4-358c067c587b admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:07.174426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fe061-ccb1-49c9-a2d4-358c067c587b admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:07.174426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fe061-ccb1-49c9-a2d4-358c067c587b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:26:07.174426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WARNING py.warnings [None req-605fe061-ccb1-49c9-a2d4-358c067c587b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:26:07.174426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: warnings.warn(msg) Sep 23 01:26:07.174426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: Sep 23 01:26:07.175108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fe061-ccb1-49c9-a2d4-358c067c587b admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:26:07.272227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 756/1498] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:26:07 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:26:07.276726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7a242-8365-4d5a-a221-970a36dc9be4 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:26:07.287254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a7a242-8365-4d5a-a221-970a36dc9be4 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:26:07.292546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a69e2-ec0e-40af-8c02-b0449263c652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:26:07.293169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a69e2-ec0e-40af-8c02-b0449263c652 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:07.293457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a69e2-ec0e-40af-8c02-b0449263c652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:07.293773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a69e2-ec0e-40af-8c02-b0449263c652 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:07.294413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a69e2-ec0e-40af-8c02-b0449263c652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:26:07.295472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: WARNING py.warnings [None req-dc0a69e2-ec0e-40af-8c02-b0449263c652 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:26:07.295472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: warnings.warn(msg) Sep 23 01:26:07.295472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: Sep 23 01:26:07.296414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a69e2-ec0e-40af-8c02-b0449263c652 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:26:07.310969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 743/1499] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:26:07.315271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fe061-ccb1-49c9-a2d4-358c067c587b admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:26:07.326587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fe061-ccb1-49c9-a2d4-358c067c587b admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:26:07.331396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1558ad-44e6-4a7d-b3d9-eadd5c218dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:26:07.332066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1558ad-44e6-4a7d-b3d9-eadd5c218dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:07.332369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1558ad-44e6-4a7d-b3d9-eadd5c218dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:07.332647 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1558ad-44e6-4a7d-b3d9-eadd5c218dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:07.333306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1558ad-44e6-4a7d-b3d9-eadd5c218dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:26:07.334588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WARNING py.warnings [None req-0f1558ad-44e6-4a7d-b3d9-eadd5c218dd2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:26:07.334588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: warnings.warn(msg) Sep 23 01:26:07.334588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: Sep 23 01:26:07.335484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1558ad-44e6-4a7d-b3d9-eadd5c218dd2 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:26:07.349552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 757/1500] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:26:07.354622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a69e2-ec0e-40af-8c02-b0449263c652 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:26:07.365293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a69e2-ec0e-40af-8c02-b0449263c652 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:26:07.371099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bb7fe-4c40-4c59-a9f6-33665a3d3578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:26:07.372100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bb7fe-4c40-4c59-a9f6-33665a3d3578 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:07.372100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bb7fe-4c40-4c59-a9f6-33665a3d3578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:07.372395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bb7fe-4c40-4c59-a9f6-33665a3d3578 admin admin] PATH_INFO: `/v3/projects/d3eed10bfbd04721b032596866b4b244/users/80743a6447c643629704a7bebdc8d732/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:07.389460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729bb7fe-4c40-4c59-a9f6-33665a3d3578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3eed10bfbd04721b032596866b4b244, user_id=80743a6447c643629704a7bebdc8d732, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:26:07.390516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: WARNING py.warnings [None req-729bb7fe-4c40-4c59-a9f6-33665a3d3578 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:26:07.390516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: warnings.warn(msg) Sep 23 01:26:07.390516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: Sep 23 01:26:07.392520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729bb7fe-4c40-4c59-a9f6-33665a3d3578 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:26:07.467030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 744/1501] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:26:07 2022] PUT /identity/v3/projects/d3eed10bfbd04721b032596866b4b244/users/80743a6447c643629704a7bebdc8d732/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:26:07.480221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a492d-c94e-45ec-b48e-b20f27462f3b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:07.480504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a492d-c94e-45ec-b48e-b20f27462f3b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:07.480831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a492d-c94e-45ec-b48e-b20f27462f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:07.512066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-e00a492d-c94e-45ec-b48e-b20f27462f3b None None] MFA Rules not processed for user `80743a6447c643629704a7bebdc8d732`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:26:07.513316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e00a492d-c94e-45ec-b48e-b20f27462f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:26:07.593534 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 758/1502] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:26:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:26:07.607904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef8a0e-0a99-4ff6-8281-3233c1256587 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:07.608190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef8a0e-0a99-4ff6-8281-3233c1256587 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:07.608466 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef8a0e-0a99-4ff6-8281-3233c1256587 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:07.609894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 745/1503] 10.222.0.64 () {54 vars in 963 bytes} [Fri Sep 23 01:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:26:07.615349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9370dc-6097-404c-9200-791623dfdfab None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:07.615600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9370dc-6097-404c-9200-791623dfdfab None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:07.615870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9370dc-6097-404c-9200-791623dfdfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:07.684458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-bc9370dc-6097-404c-9200-791623dfdfab None None] MFA Rules not processed for user `51ae9d865bb4442490ca61b18d1f7ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:26:07.685694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bc9370dc-6097-404c-9200-791623dfdfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:26:07.766160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 759/1504] 10.222.0.64 () {58 vars in 1060 bytes} [Fri Sep 23 01:26:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:26:07.774489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef8a0e-0a99-4ff6-8281-3233c1256587 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:26:07.785168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef8a0e-0a99-4ff6-8281-3233c1256587 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:26:07.788835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb4738-eea9-40c5-be6c-6eb9be74fb6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:26:07.789430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb4738-eea9-40c5-be6c-6eb9be74fb6f service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:07.790298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb4738-eea9-40c5-be6c-6eb9be74fb6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:07.790575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb4738-eea9-40c5-be6c-6eb9be74fb6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:07.798926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb4738-eea9-40c5-be6c-6eb9be74fb6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:26:07.805251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb4738-eea9-40c5-be6c-6eb9be74fb6f service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:26:07.809611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 746/1505] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:26:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:26:07.861521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf029224-8232-42f2-b3a0-76a840a0d613 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:07.861814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf029224-8232-42f2-b3a0-76a840a0d613 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:07.862097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf029224-8232-42f2-b3a0-76a840a0d613 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:07.863418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 760/1506] 10.222.0.64 () {52 vars in 830 bytes} [Fri Sep 23 01:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:26:07.868006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb4738-eea9-40c5-be6c-6eb9be74fb6f service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:26:07.931686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb4738-eea9-40c5-be6c-6eb9be74fb6f service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:26:07.936456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:26:07.937104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:07.937379 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:07.937658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] PATH_INFO: `/v3/users/80743a6447c643629704a7bebdc8d732` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:07.938956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=80743a6447c643629704a7bebdc8d732)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:26:07.940006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: WARNING py.warnings [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:26:07.940006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: warnings.warn(msg) Sep 23 01:26:07.940006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: Sep 23 01:26:07.941228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:26:08.139911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80743a6447c643629704a7bebdc8d732'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:26:08.208855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80743a6447c643629704a7bebdc8d732'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:26:08.209620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80743a6447c643629704a7bebdc8d732'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:26:08.241360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80743a6447c643629704a7bebdc8d732'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:26:08.242584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80743a6447c643629704a7bebdc8d732'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:26:08.255025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:26:08.267141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 747/1507] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:26:07 2022] DELETE /identity/v3/users/80743a6447c643629704a7bebdc8d732 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:26:08.271996 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd679f9-9524-455a-8ec8-2233ad6a5b8d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:08.274971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd679f9-9524-455a-8ec8-2233ad6a5b8d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:08.274971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd679f9-9524-455a-8ec8-2233ad6a5b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:08.294814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-fbd679f9-9524-455a-8ec8-2233ad6a5b8d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:26:08.297995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-fbd679f9-9524-455a-8ec8-2233ad6a5b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:26:08.306448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:26:08.362259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 761/1508] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:26:08.377335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:26:08.377335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:26:08.466435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16442f-5500-4942-a647-675b387cd9fb admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:26:08.470299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac28ea-e007-4308-9ec6-8e29f8f2d0c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:26:08.470928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac28ea-e007-4308-9ec6-8e29f8f2d0c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:08.471210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac28ea-e007-4308-9ec6-8e29f8f2d0c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:08.471488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac28ea-e007-4308-9ec6-8e29f8f2d0c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:08.481748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac28ea-e007-4308-9ec6-8e29f8f2d0c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:26:08.483221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac28ea-e007-4308-9ec6-8e29f8f2d0c3 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:26:08.493870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 748/1509] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:26:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:26:09.142119 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd679f9-9524-455a-8ec8-2233ad6a5b8d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:26:09.143892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-fbd679f9-9524-455a-8ec8-2233ad6a5b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:26:09.175031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd679f9-9524-455a-8ec8-2233ad6a5b8d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:26:09.175031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:26:09.175031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:26:09.175031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:26:09.175473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] PATH_INFO: `/v3/projects/d3eed10bfbd04721b032596866b4b244` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:26:09.175473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3eed10bfbd04721b032596866b4b244)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:26:09.175473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WARNING py.warnings [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:26:09.175473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: warnings.warn(msg) Sep 23 01:26:09.175473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: Sep 23 01:26:09.175473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:26:09.376578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3eed10bfbd04721b032596866b4b244', 'request_id': 'req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:26:09.397839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3eed10bfbd04721b032596866b4b244', 'request_id': 'req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:26:09.488758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3eed10bfbd04721b032596866b4b244', 'request_id': 'req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:26:09.501444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] The token cache is being invalidate because project d3eed10bfbd04721b032596866b4b244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:26:09.501733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:26:09.502408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2de43b96-f0e0-40cd-a33f-68b8ef20f7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:26:09.504121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 762/1510] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:26:09 2022] DELETE /identity/v3/projects/d3eed10bfbd04721b032596866b4b244 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:26:18.488807 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=38 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:26:19.497980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:27:42.321316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda54814-8fdb-45c8-9cdc-a3907ebd8e5e None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:42.322498 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda54814-8fdb-45c8-9cdc-a3907ebd8e5e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:42.322498 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda54814-8fdb-45c8-9cdc-a3907ebd8e5e None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:42.323967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:27:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:27:42.329330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518dd2-1c28-4519-b6ea-a17669a94418 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:42.329812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518dd2-1c28-4519-b6ea-a17669a94418 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:42.329928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9518dd2-1c28-4519-b6ea-a17669a94418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:42.350815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f9518dd2-1c28-4519-b6ea-a17669a94418 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:27:42.352070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f9518dd2-1c28-4519-b6ea-a17669a94418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:27:42.420993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:27:42.576987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda54814-8fdb-45c8-9cdc-a3907ebd8e5e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:27:42.578810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-eda54814-8fdb-45c8-9cdc-a3907ebd8e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:27:42.682323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda54814-8fdb-45c8-9cdc-a3907ebd8e5e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:27:42.687870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37465cdd-7f0f-4b55-8257-a65423fabfd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:27:42.688476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37465cdd-7f0f-4b55-8257-a65423fabfd9 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:42.688753 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37465cdd-7f0f-4b55-8257-a65423fabfd9 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:42.689024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37465cdd-7f0f-4b55-8257-a65423fabfd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:42.697813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37465cdd-7f0f-4b55-8257-a65423fabfd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:27:42.699401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37465cdd-7f0f-4b55-8257-a65423fabfd9 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:27:42.709220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 750/1513] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:27:43.806008 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71850631-0acd-49a5-9dc7-5274f8ead401 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:43.806598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71850631-0acd-49a5-9dc7-5274f8ead401 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:43.806707 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71850631-0acd-49a5-9dc7-5274f8ead401 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:43.808344 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:27:43.814776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f56074-dcc1-4161-abe8-853f12522da5 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:43.815281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f56074-dcc1-4161-abe8-853f12522da5 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:43.815401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f56074-dcc1-4161-abe8-853f12522da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:43.835494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-48f56074-dcc1-4161-abe8-853f12522da5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:27:43.836992 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-48f56074-dcc1-4161-abe8-853f12522da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:27:43.845702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:27:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:27:43.997587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71850631-0acd-49a5-9dc7-5274f8ead401 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:27:44.011870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71850631-0acd-49a5-9dc7-5274f8ead401 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:27:44.017310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427861ec-df42-4826-a73f-a7753f1f92e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:27:44.017915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427861ec-df42-4826-a73f-a7753f1f92e6 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:44.018198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427861ec-df42-4826-a73f-a7753f1f92e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:44.018474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427861ec-df42-4826-a73f-a7753f1f92e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:44.027444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427861ec-df42-4826-a73f-a7753f1f92e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:27:44.029079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427861ec-df42-4826-a73f-a7753f1f92e6 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:27:44.038781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 765/1516] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:27:44.293403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7051c99-68eb-484f-8a44-0590ba778a0c None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:44.293688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7051c99-68eb-484f-8a44-0590ba778a0c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:44.293972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7051c99-68eb-484f-8a44-0590ba778a0c None None] PATH_INFO: `/v3` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:44.295332 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 752/1517] 10.222.0.81 () {54 vars in 925 bytes} [Fri Sep 23 01:27:44 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:27:44.301751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49246f-0005-4068-8886-2b198b78bf19 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:44.302123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49246f-0005-4068-8886-2b198b78bf19 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:44.302329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49246f-0005-4068-8886-2b198b78bf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:44.352224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9c49246f-0005-4068-8886-2b198b78bf19 None None] MFA Rules not processed for user `9f73bca3896440258430cfce042faa42`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:27:44.353901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9c49246f-0005-4068-8886-2b198b78bf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:27:44.426468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 766/1518] 10.222.0.81 () {58 vars in 1015 bytes} [Fri Sep 23 01:27:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:27:53.437358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bd061-77d7-44b9-8dc8-05caf7f62b7d None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:53.438093 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bd061-77d7-44b9-8dc8-05caf7f62b7d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:53.438093 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439bd061-77d7-44b9-8dc8-05caf7f62b7d None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:53.439366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 753/1519] 10.222.0.64 () {54 vars in 961 bytes} [Fri Sep 23 01:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:27:53.444700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9759ce6-2fed-46a8-b51f-afbe5508ffba None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:53.445217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9759ce6-2fed-46a8-b51f-afbe5508ffba None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:53.445492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9759ce6-2fed-46a8-b51f-afbe5508ffba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:53.466356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a9759ce6-2fed-46a8-b51f-afbe5508ffba None None] MFA Rules not processed for user `9f73bca3896440258430cfce042faa42`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:27:53.467657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a9759ce6-2fed-46a8-b51f-afbe5508ffba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:27:53.476204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 767/1520] 10.222.0.64 () {58 vars in 1058 bytes} [Fri Sep 23 01:27:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:27:53.484610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bd061-77d7-44b9-8dc8-05caf7f62b7d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:27:53.498693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439bd061-77d7-44b9-8dc8-05caf7f62b7d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:27:53.503778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a79bf6-230f-4ca0-a8a6-55773112ddf7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:27:53.504382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a79bf6-230f-4ca0-a8a6-55773112ddf7 service placement] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:53.504663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a79bf6-230f-4ca0-a8a6-55773112ddf7 service placement] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:53.504939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a79bf6-230f-4ca0-a8a6-55773112ddf7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:53.513660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a79bf6-230f-4ca0-a8a6-55773112ddf7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:27:53.514959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a79bf6-230f-4ca0-a8a6-55773112ddf7 service placement] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:27:53.524835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 754/1521] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:27:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:27:55.552818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cea0b6-ab5c-40bf-a5d1-432c84e2452c None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:55.553395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cea0b6-ab5c-40bf-a5d1-432c84e2452c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:55.553395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cea0b6-ab5c-40bf-a5d1-432c84e2452c None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:55.554752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:27:55.561392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b3d5f-e5ed-43d5-8fcc-42abf9595df1 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:55.561885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b3d5f-e5ed-43d5-8fcc-42abf9595df1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:55.561999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b3d5f-e5ed-43d5-8fcc-42abf9595df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:55.581144 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8f7b3d5f-e5ed-43d5-8fcc-42abf9595df1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:27:55.582367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8f7b3d5f-e5ed-43d5-8fcc-42abf9595df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:27:55.591168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:27:55.747358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cea0b6-ab5c-40bf-a5d1-432c84e2452c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:27:55.762369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cea0b6-ab5c-40bf-a5d1-432c84e2452c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:27:55.768064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f27ee9-bba1-4a19-8f8b-30e9c4c29fba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:27:55.768669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27ee9-bba1-4a19-8f8b-30e9c4c29fba service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:55.768949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27ee9-bba1-4a19-8f8b-30e9c4c29fba service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:55.769222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f27ee9-bba1-4a19-8f8b-30e9c4c29fba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:55.778628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f27ee9-bba1-4a19-8f8b-30e9c4c29fba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:27:55.780317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f27ee9-bba1-4a19-8f8b-30e9c4c29fba service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:27:55.790171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 769/1524] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:27:56.877308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e2058-d356-4434-bd3a-0243f7624545 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:56.877861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e2058-d356-4434-bd3a-0243f7624545 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:56.877988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e2058-d356-4434-bd3a-0243f7624545 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:56.879295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 23 01:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:27:56.886349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d4ffb-06c1-4e45-b3be-8f24491451f4 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:56.886855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d4ffb-06c1-4e45-b3be-8f24491451f4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:56.886942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d4ffb-06c1-4e45-b3be-8f24491451f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:56.906496 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-4a4d4ffb-06c1-4e45-b3be-8f24491451f4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:27:56.907763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4a4d4ffb-06c1-4e45-b3be-8f24491451f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:27:56.915987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 23 01:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:27:57.074788 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e2058-d356-4434-bd3a-0243f7624545 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:27:57.089996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e2058-d356-4434-bd3a-0243f7624545 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:27:57.095589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285991e-3994-456f-a03d-33b29805e25a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:27:57.096182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285991e-3994-456f-a03d-33b29805e25a service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:27:57.096457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285991e-3994-456f-a03d-33b29805e25a service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:27:57.096729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285991e-3994-456f-a03d-33b29805e25a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:27:57.107185 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285991e-3994-456f-a03d-33b29805e25a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:27:57.108832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285991e-3994-456f-a03d-33b29805e25a service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:27:57.118730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 757/1527] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:28:01.183949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b893a-a277-4be5-aee9-68aade578f9e None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:01.184530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b893a-a277-4be5-aee9-68aade578f9e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:01.184530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b893a-a277-4be5-aee9-68aade578f9e None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:01.185959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:28:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:28:01.191930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f41b30-9613-45c2-9a4f-45bfb0558ece None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:01.192412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f41b30-9613-45c2-9a4f-45bfb0558ece None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:01.192524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f41b30-9613-45c2-9a4f-45bfb0558ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:01.213495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b6f41b30-9613-45c2-9a4f-45bfb0558ece None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:28:01.215631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b6f41b30-9613-45c2-9a4f-45bfb0558ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:28:01.223643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:28:01.510905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b893a-a277-4be5-aee9-68aade578f9e None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:28:01.512701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bd9b893a-a277-4be5-aee9-68aade578f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:28:01.545079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:28:01.574638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b893a-a277-4be5-aee9-68aade578f9e None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:28:01.580764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05998add-dc85-4ed6-9d93-c231b6c2bd22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:28:01.581388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05998add-dc85-4ed6-9d93-c231b6c2bd22 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:01.581666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05998add-dc85-4ed6-9d93-c231b6c2bd22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:01.581936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05998add-dc85-4ed6-9d93-c231b6c2bd22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:01.592476 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05998add-dc85-4ed6-9d93-c231b6c2bd22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:28:01.593911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05998add-dc85-4ed6-9d93-c231b6c2bd22 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:28:01.603681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 772/1530] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:28:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:28:03.774460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163fc3e-4ddf-44e6-b75f-1c3c91dbfc43 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:03.775041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163fc3e-4ddf-44e6-b75f-1c3c91dbfc43 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:03.775041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163fc3e-4ddf-44e6-b75f-1c3c91dbfc43 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:03.776430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:28:03.783382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4123db0-61c1-44d8-addf-638fc3593a27 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:03.783948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4123db0-61c1-44d8-addf-638fc3593a27 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:03.784139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4123db0-61c1-44d8-addf-638fc3593a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:03.804224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a4123db0-61c1-44d8-addf-638fc3593a27 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:28:03.805430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a4123db0-61c1-44d8-addf-638fc3593a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:28:03.813546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:28:04.118820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163fc3e-4ddf-44e6-b75f-1c3c91dbfc43 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:28:04.130445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163fc3e-4ddf-44e6-b75f-1c3c91dbfc43 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:28:04.134101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8fc3-67e8-427b-ac10-3f141a4a4392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:28:04.134697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8fc3-67e8-427b-ac10-3f141a4a4392 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:04.134980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8fc3-67e8-427b-ac10-3f141a4a4392 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:04.135258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8fc3-67e8-427b-ac10-3f141a4a4392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:04.145210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8fc3-67e8-427b-ac10-3f141a4a4392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:28:04.146814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8fc3-67e8-427b-ac10-3f141a4a4392 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:28:04.157574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 760/1533] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:28:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:28:07.997790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9decc8-616b-4e8b-983d-2c05ff9c1f85 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:08.030767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9decc8-616b-4e8b-983d-2c05ff9c1f85 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:08.030767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9decc8-616b-4e8b-983d-2c05ff9c1f85 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:08.030767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:28:08.031114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc33d-0adf-4a57-b4ef-6eca836dc4f6 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:08.031114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc33d-0adf-4a57-b4ef-6eca836dc4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:08.031114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc33d-0adf-4a57-b4ef-6eca836dc4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:08.031114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-7d5bc33d-0adf-4a57-b4ef-6eca836dc4f6 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:28:08.031114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7d5bc33d-0adf-4a57-b4ef-6eca836dc4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:28:08.036126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:28:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:28:08.342829 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9decc8-616b-4e8b-983d-2c05ff9c1f85 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:28:08.344630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-db9decc8-616b-4e8b-983d-2c05ff9c1f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:28:08.408185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9decc8-616b-4e8b-983d-2c05ff9c1f85 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:28:08.412498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d073fc-75d4-4cd2-b575-c7f2b19e4123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:28:08.413478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d073fc-75d4-4cd2-b575-c7f2b19e4123 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:08.413760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d073fc-75d4-4cd2-b575-c7f2b19e4123 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:08.414086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d073fc-75d4-4cd2-b575-c7f2b19e4123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:08.425639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d073fc-75d4-4cd2-b575-c7f2b19e4123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:28:08.427118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d073fc-75d4-4cd2-b575-c7f2b19e4123 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:28:08.438398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 775/1536] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:28:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:28:10.659427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c7bf7-d67a-4365-819b-6b3ac3f55773 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:10.660094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c7bf7-d67a-4365-819b-6b3ac3f55773 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:10.660204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c7bf7-d67a-4365-819b-6b3ac3f55773 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:10.661593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 23 01:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:28:10.667017 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085425bb-abf7-4de3-9f15-daf717f33348 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:10.668374 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085425bb-abf7-4de3-9f15-daf717f33348 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:10.668374 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085425bb-abf7-4de3-9f15-daf717f33348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:10.689948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-085425bb-abf7-4de3-9f15-daf717f33348 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:28:10.691243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-085425bb-abf7-4de3-9f15-daf717f33348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:28:10.699390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 23 01:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:28:11.019021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c7bf7-d67a-4365-819b-6b3ac3f55773 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:28:11.021791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1c5c7bf7-d67a-4365-819b-6b3ac3f55773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:28:11.030317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=77 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:28:11.037429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c7bf7-d67a-4365-819b-6b3ac3f55773 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:28:11.043507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb5964-3574-4228-b782-da2fb87032ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:28:11.044154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb5964-3574-4228-b782-da2fb87032ae service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:28:11.044437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb5964-3574-4228-b782-da2fb87032ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:28:11.044735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb5964-3574-4228-b782-da2fb87032ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:28:11.054403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb5964-3574-4228-b782-da2fb87032ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:28:11.055896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb5964-3574-4228-b782-da2fb87032ae service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:28:11.066247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 763/1539] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:28:20.684351 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=47 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:28:21.149303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:35.375261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3a554-d3c7-4066-a19b-79e8d24b7755 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.375945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3a554-d3c7-4066-a19b-79e8d24b7755 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.375945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3a554-d3c7-4066-a19b-79e8d24b7755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.400842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae543ab-2d50-4f91-8e6e-f6317c053afd None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.402026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae543ab-2d50-4f91-8e6e-f6317c053afd None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.402712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae543ab-2d50-4f91-8e6e-f6317c053afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.407388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-bfa3a554-d3c7-4066-a19b-79e8d24b7755 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:35.407388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bfa3a554-d3c7-4066-a19b-79e8d24b7755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:35.427016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 777/1540] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:35.434624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-3ae543ab-2d50-4f91-8e6e-f6317c053afd None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:35.436500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3ae543ab-2d50-4f91-8e6e-f6317c053afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:35.451906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 764/1541] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:35.457790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7bc1b-12c2-44c5-9107-9d935b6461d3 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.458064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7bc1b-12c2-44c5-9107-9d935b6461d3 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.458339 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7bc1b-12c2-44c5-9107-9d935b6461d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.463096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df7da0-3bf9-4dc3-8ea6-c622a6000ca8 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.464080 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df7da0-3bf9-4dc3-8ea6-c622a6000ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.464600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df7da0-3bf9-4dc3-8ea6-c622a6000ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.492097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6aa7bc1b-12c2-44c5-9107-9d935b6461d3 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:35.493402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6aa7bc1b-12c2-44c5-9107-9d935b6461d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:35.495579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-60df7da0-3bf9-4dc3-8ea6-c622a6000ca8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:35.497252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-60df7da0-3bf9-4dc3-8ea6-c622a6000ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:35.517281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 765/1542] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:29:35.517613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 778/1543] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:35.522590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7bc1b-12c2-44c5-9107-9d935b6461d3 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:35.522801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df7da0-3bf9-4dc3-8ea6-c622a6000ca8 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:35.538458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df7da0-3bf9-4dc3-8ea6-c622a6000ca8 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:35.538695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7bc1b-12c2-44c5-9107-9d935b6461d3 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:35.557380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00034f53-0760-4a6d-985a-4aa263f57bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:35.558016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00034f53-0760-4a6d-985a-4aa263f57bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.558301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00034f53-0760-4a6d-985a-4aa263f57bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.558577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00034f53-0760-4a6d-985a-4aa263f57bd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.558820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a43f9-7779-4e56-ab4f-b355eddfc2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:35.559240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00034f53-0760-4a6d-985a-4aa263f57bd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:29:35.559633 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00034f53-0760-4a6d-985a-4aa263f57bd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:35.560230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a43f9-7779-4e56-ab4f-b355eddfc2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.560742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a43f9-7779-4e56-ab4f-b355eddfc2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.561186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00034f53-0760-4a6d-985a-4aa263f57bd1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:35.561463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a43f9-7779-4e56-ab4f-b355eddfc2c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.563115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a43f9-7779-4e56-ab4f-b355eddfc2c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:29:35.566623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a43f9-7779-4e56-ab4f-b355eddfc2c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:35.568116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: WARNING py.warnings [None req-0a8a43f9-7779-4e56-ab4f-b355eddfc2c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:29:35.568116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: warnings.warn(msg) Sep 23 01:29:35.568116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: Sep 23 01:29:35.569440 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a43f9-7779-4e56-ab4f-b355eddfc2c5 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:35.585896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 779/1544] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:29:35.592534 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c235b6-5938-45b8-a2d2-8cd33d0940a2 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.592877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c235b6-5938-45b8-a2d2-8cd33d0940a2 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.593099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c235b6-5938-45b8-a2d2-8cd33d0940a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.593552 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 766/1545] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:29:35.596274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a43f9-7779-4e56-ab4f-b355eddfc2c5 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:35.614355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a43f9-7779-4e56-ab4f-b355eddfc2c5 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:35.628099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8227e81-cabe-42cc-bc2b-573e2acf0afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:35.629127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8227e81-cabe-42cc-bc2b-573e2acf0afe admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.629651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8227e81-cabe-42cc-bc2b-573e2acf0afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.630391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8227e81-cabe-42cc-bc2b-573e2acf0afe admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.631280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8227e81-cabe-42cc-bc2b-573e2acf0afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:35.632741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-05c235b6-5938-45b8-a2d2-8cd33d0940a2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:35.633893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8227e81-cabe-42cc-bc2b-573e2acf0afe admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:35.634107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-05c235b6-5938-45b8-a2d2-8cd33d0940a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:35.657353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 780/1546] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:35.657589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c235b6-5938-45b8-a2d2-8cd33d0940a2 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:35.674938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c235b6-5938-45b8-a2d2-8cd33d0940a2 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:35.696755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2943b3e9-dad1-4688-a97d-3b98a3fa7254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:35.697403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943b3e9-dad1-4688-a97d-3b98a3fa7254 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.697692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943b3e9-dad1-4688-a97d-3b98a3fa7254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.697970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2943b3e9-dad1-4688-a97d-3b98a3fa7254 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.698663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943b3e9-dad1-4688-a97d-3b98a3fa7254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:35.699777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WARNING py.warnings [None req-2943b3e9-dad1-4688-a97d-3b98a3fa7254 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:29:35.699777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: warnings.warn(msg) Sep 23 01:29:35.699777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: Sep 23 01:29:35.701053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2943b3e9-dad1-4688-a97d-3b98a3fa7254 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:35.791688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 767/1547] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:35.793562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3db85d-47e7-42aa-ad8a-a733dcda351d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.794032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3db85d-47e7-42aa-ad8a-a733dcda351d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.794480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3db85d-47e7-42aa-ad8a-a733dcda351d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.814025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 781/1548] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:35.815187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb475e5-9b87-41e5-9ece-b065a6895a0a None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.815479 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb475e5-9b87-41e5-9ece-b065a6895a0a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.815785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb475e5-9b87-41e5-9ece-b065a6895a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.820886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-0d3db85d-47e7-42aa-ad8a-a733dcda351d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:35.822627 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0d3db85d-47e7-42aa-ad8a-a733dcda351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:35.847488 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-7cb475e5-9b87-41e5-9ece-b065a6895a0a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:35.849669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7cb475e5-9b87-41e5-9ece-b065a6895a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:35.904268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 768/1549] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:35.905109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda92288-0f78-4a94-8fc1-9474e58cbcca None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.905496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda92288-0f78-4a94-8fc1-9474e58cbcca None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.905669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda92288-0f78-4a94-8fc1-9474e58cbcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.927146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-bda92288-0f78-4a94-8fc1-9474e58cbcca None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:35.928402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-bda92288-0f78-4a94-8fc1-9474e58cbcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:35.936402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 769/1550] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:35.938248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2789bc-887e-46c0-a321-d6dd091dfd08 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.938526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2789bc-887e-46c0-a321-d6dd091dfd08 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.938830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2789bc-887e-46c0-a321-d6dd091dfd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.951849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 782/1551] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:35.953288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb475e5-9b87-41e5-9ece-b065a6895a0a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:35.964631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-1c2789bc-887e-46c0-a321-d6dd091dfd08 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:35.965871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1c2789bc-887e-46c0-a321-d6dd091dfd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:35.967320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb475e5-9b87-41e5-9ece-b065a6895a0a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:35.972459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8de55-e4c6-45c5-bc34-aa971e4d6073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:35.973119 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8de55-e4c6-45c5-bc34-aa971e4d6073 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.973491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8de55-e4c6-45c5-bc34-aa971e4d6073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.973706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8de55-e4c6-45c5-bc34-aa971e4d6073 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.974335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8de55-e4c6-45c5-bc34-aa971e4d6073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:35.974457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 770/1552] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:35.979927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2789bc-887e-46c0-a321-d6dd091dfd08 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:35.980081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8de55-e4c6-45c5-bc34-aa971e4d6073 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:35.990566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2789bc-887e-46c0-a321-d6dd091dfd08 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:35.996129 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287a8d0-e75f-432e-966f-bd485ec58a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:35.996754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287a8d0-e75f-432e-966f-bd485ec58a43 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:35.997048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287a8d0-e75f-432e-966f-bd485ec58a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:35.997414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287a8d0-e75f-432e-966f-bd485ec58a43 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:35.997978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287a8d0-e75f-432e-966f-bd485ec58a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:35.999027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: WARNING py.warnings [None req-d287a8d0-e75f-432e-966f-bd485ec58a43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:29:35.999027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: warnings.warn(msg) Sep 23 01:29:35.999027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: Sep 23 01:29:36.000856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287a8d0-e75f-432e-966f-bd485ec58a43 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.081194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 771/1553] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:36.081905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 783/1554] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:35 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:36.082489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f763af2-eb46-4653-9d30-10ad180aa12f None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.082782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f763af2-eb46-4653-9d30-10ad180aa12f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.083052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f763af2-eb46-4653-9d30-10ad180aa12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.083216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a29e-ae15-43ae-9cb7-e6b24ebb7d5b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.083599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a29e-ae15-43ae-9cb7-e6b24ebb7d5b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.083944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140a29e-ae15-43ae-9cb7-e6b24ebb7d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.104705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-6f763af2-eb46-4653-9d30-10ad180aa12f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:36.105791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6f763af2-eb46-4653-9d30-10ad180aa12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:36.105991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b140a29e-ae15-43ae-9cb7-e6b24ebb7d5b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:36.107198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b140a29e-ae15-43ae-9cb7-e6b24ebb7d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:36.114567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 772/1555] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:36.115922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d35e-c53c-420c-a4e7-6a882b1632c5 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.116129 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d35e-c53c-420c-a4e7-6a882b1632c5 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.116402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d35e-c53c-420c-a4e7-6a882b1632c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.126294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 784/1556] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:36.126294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff3e78-4c5d-4b3a-bcbc-4884d58b414f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.126294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff3e78-4c5d-4b3a-bcbc-4884d58b414f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.126294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff3e78-4c5d-4b3a-bcbc-4884d58b414f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.143255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8704d35e-c53c-420c-a4e7-6a882b1632c5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:36.144585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8704d35e-c53c-420c-a4e7-6a882b1632c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:36.148952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-edff3e78-4c5d-4b3a-bcbc-4884d58b414f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:36.153876 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-edff3e78-4c5d-4b3a-bcbc-4884d58b414f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:36.159661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 773/1557] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:36.160453 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d35e-c53c-420c-a4e7-6a882b1632c5 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.163745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 785/1558] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:36.166158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0601c-948a-432b-bfcc-45fc66bfae2b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.166646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0601c-948a-432b-bfcc-45fc66bfae2b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.167127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0601c-948a-432b-bfcc-45fc66bfae2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.173557 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d35e-c53c-420c-a4e7-6a882b1632c5 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.178339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ac51-11d7-43ce-8748-34120464e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.178952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ac51-11d7-43ce-8748-34120464e316 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.179259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ac51-11d7-43ce-8748-34120464e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.179539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ac51-11d7-43ce-8748-34120464e316 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.180256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3ac51-11d7-43ce-8748-34120464e316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:29:36.180565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3ac51-11d7-43ce-8748-34120464e316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.182193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3ac51-11d7-43ce-8748-34120464e316 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.195379 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 774/1559] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:29:36.196313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ac51-11d7-43ce-8748-34120464e316 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.201973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b4a0601c-948a-432b-bfcc-45fc66bfae2b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:36.204683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b4a0601c-948a-432b-bfcc-45fc66bfae2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:36.208526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ac51-11d7-43ce-8748-34120464e316 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.214885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95945b56-453e-4021-9128-c2bf51b2570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.219831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95945b56-453e-4021-9128-c2bf51b2570c admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.219831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95945b56-453e-4021-9128-c2bf51b2570c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.219831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95945b56-453e-4021-9128-c2bf51b2570c admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.220362 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 786/1560] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:36.220515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95945b56-453e-4021-9128-c2bf51b2570c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.220625 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0601c-948a-432b-bfcc-45fc66bfae2b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.220724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95945b56-453e-4021-9128-c2bf51b2570c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.229775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0601c-948a-432b-bfcc-45fc66bfae2b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.235190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 775/1561] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:36.235190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f410817-fc3d-4b04-b1ef-76a546abfe11 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.235190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f410817-fc3d-4b04-b1ef-76a546abfe11 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.235190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f410817-fc3d-4b04-b1ef-76a546abfe11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.235791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966493f0-7e0b-4d96-a728-b329ab803a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.236467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966493f0-7e0b-4d96-a728-b329ab803a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.237005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966493f0-7e0b-4d96-a728-b329ab803a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.237005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966493f0-7e0b-4d96-a728-b329ab803a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.237858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966493f0-7e0b-4d96-a728-b329ab803a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.239857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966493f0-7e0b-4d96-a728-b329ab803a25 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.256114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 787/1562] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:36.256114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578afdfb-766b-4bb1-9240-38703dde7c6c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.256114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578afdfb-766b-4bb1-9240-38703dde7c6c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.256114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578afdfb-766b-4bb1-9240-38703dde7c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.256606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-1f410817-fc3d-4b04-b1ef-76a546abfe11 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:36.258070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1f410817-fc3d-4b04-b1ef-76a546abfe11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:36.266197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 776/1563] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:36.267121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f410817-fc3d-4b04-b1ef-76a546abfe11 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.277202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-578afdfb-766b-4bb1-9240-38703dde7c6c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:36.277422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f410817-fc3d-4b04-b1ef-76a546abfe11 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.278465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-578afdfb-766b-4bb1-9240-38703dde7c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:36.283994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631251f-d8b2-4ad0-8bcf-f4cb0f4ee6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.284606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631251f-d8b2-4ad0-8bcf-f4cb0f4ee6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.284885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631251f-d8b2-4ad0-8bcf-f4cb0f4ee6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.285152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631251f-d8b2-4ad0-8bcf-f4cb0f4ee6ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.285800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631251f-d8b2-4ad0-8bcf-f4cb0f4ee6ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:29:36.286083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631251f-d8b2-4ad0-8bcf-f4cb0f4ee6ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.287736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631251f-d8b2-4ad0-8bcf-f4cb0f4ee6ad admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.287916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 788/1564] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:36.288869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578afdfb-766b-4bb1-9240-38703dde7c6c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.299575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578afdfb-766b-4bb1-9240-38703dde7c6c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.301086 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 777/1565] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:29:36.301976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631251f-d8b2-4ad0-8bcf-f4cb0f4ee6ad admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.304518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a316dc7-eea7-4d33-a11d-514ea6d173b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.305157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a316dc7-eea7-4d33-a11d-514ea6d173b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.305440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a316dc7-eea7-4d33-a11d-514ea6d173b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.305714 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a316dc7-eea7-4d33-a11d-514ea6d173b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.306387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a316dc7-eea7-4d33-a11d-514ea6d173b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:29:36.306688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a316dc7-eea7-4d33-a11d-514ea6d173b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.308358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a316dc7-eea7-4d33-a11d-514ea6d173b9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.314727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631251f-d8b2-4ad0-8bcf-f4cb0f4ee6ad admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.320688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d7e34-cf5f-4a88-9260-50a62ccf89fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.321062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d7e34-cf5f-4a88-9260-50a62ccf89fd admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.321345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d7e34-cf5f-4a88-9260-50a62ccf89fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.321624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d7e34-cf5f-4a88-9260-50a62ccf89fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.322403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d7e34-cf5f-4a88-9260-50a62ccf89fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.322960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 789/1566] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:29:36.324708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d7e34-cf5f-4a88-9260-50a62ccf89fd admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.324885 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a316dc7-eea7-4d33-a11d-514ea6d173b9 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.337665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a316dc7-eea7-4d33-a11d-514ea6d173b9 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.343908 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae9308-79ef-4000-b706-6aa84b93cc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.344545 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae9308-79ef-4000-b706-6aa84b93cc0e admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.344835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae9308-79ef-4000-b706-6aa84b93cc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.345236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae9308-79ef-4000-b706-6aa84b93cc0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.345957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae9308-79ef-4000-b706-6aa84b93cc0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:29:36.346395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae9308-79ef-4000-b706-6aa84b93cc0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.348207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae9308-79ef-4000-b706-6aa84b93cc0e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.362366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 790/1567] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:29:36.367037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae9308-79ef-4000-b706-6aa84b93cc0e admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.374019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae9308-79ef-4000-b706-6aa84b93cc0e admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.378992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4e8cc-bee2-4020-87c9-0f22b0a92359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.379704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4e8cc-bee2-4020-87c9-0f22b0a92359 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.379982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4e8cc-bee2-4020-87c9-0f22b0a92359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.380255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4e8cc-bee2-4020-87c9-0f22b0a92359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.381233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4e8cc-bee2-4020-87c9-0f22b0a92359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.382805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4e8cc-bee2-4020-87c9-0f22b0a92359 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.397778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 791/1568] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:36.398847 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4e8cc-bee2-4020-87c9-0f22b0a92359 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.410953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4e8cc-bee2-4020-87c9-0f22b0a92359 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.415878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6121f-f294-4612-9f6b-829b2bb08c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.416501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6121f-f294-4612-9f6b-829b2bb08c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.416785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6121f-f294-4612-9f6b-829b2bb08c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.417061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6121f-f294-4612-9f6b-829b2bb08c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.417717 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6121f-f294-4612-9f6b-829b2bb08c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.419322 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6121f-f294-4612-9f6b-829b2bb08c1c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.431061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 792/1569] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:36.431962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6121f-f294-4612-9f6b-829b2bb08c1c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.437735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 778/1570] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:36.438746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d7e34-cf5f-4a88-9260-50a62ccf89fd admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.440306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:36.445195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:36.445703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6121f-f294-4612-9f6b-829b2bb08c1c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.506781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613c3bc-b476-4640-82fa-1b58ca0f4018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.507410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613c3bc-b476-4640-82fa-1b58ca0f4018 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.507741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613c3bc-b476-4640-82fa-1b58ca0f4018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.508019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613c3bc-b476-4640-82fa-1b58ca0f4018 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.508681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613c3bc-b476-4640-82fa-1b58ca0f4018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:29:36.508990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613c3bc-b476-4640-82fa-1b58ca0f4018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.510644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613c3bc-b476-4640-82fa-1b58ca0f4018 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.514881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d7e34-cf5f-4a88-9260-50a62ccf89fd admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.521082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056c41a-b56f-43e1-b5f2-fd3bbd0b4308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.521721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056c41a-b56f-43e1-b5f2-fd3bbd0b4308 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.522008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056c41a-b56f-43e1-b5f2-fd3bbd0b4308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.522285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056c41a-b56f-43e1-b5f2-fd3bbd0b4308 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.522999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056c41a-b56f-43e1-b5f2-fd3bbd0b4308 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:29:36.523351 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056c41a-b56f-43e1-b5f2-fd3bbd0b4308 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.524509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 793/1571] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:29:36.525378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613c3bc-b476-4640-82fa-1b58ca0f4018 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.532457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056c41a-b56f-43e1-b5f2-fd3bbd0b4308 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.536149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613c3bc-b476-4640-82fa-1b58ca0f4018 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.541888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b3949-144b-4745-863e-1d9b8a559027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.542500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b3949-144b-4745-863e-1d9b8a559027 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.542778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b3949-144b-4745-863e-1d9b8a559027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.543055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b3949-144b-4745-863e-1d9b8a559027 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.543735 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b3949-144b-4745-863e-1d9b8a559027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.553256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 779/1572] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:29:36.553481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b3949-144b-4745-863e-1d9b8a559027 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.553595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056c41a-b56f-43e1-b5f2-fd3bbd0b4308 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.557601 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056c41a-b56f-43e1-b5f2-fd3bbd0b4308 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.564661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6831ef2-8a6c-4f06-9080-a2638e750eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.564661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6831ef2-8a6c-4f06-9080-a2638e750eb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.564661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6831ef2-8a6c-4f06-9080-a2638e750eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.565014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6831ef2-8a6c-4f06-9080-a2638e750eb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.565640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6831ef2-8a6c-4f06-9080-a2638e750eb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.567720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6831ef2-8a6c-4f06-9080-a2638e750eb2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.633077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 794/1573] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:36.634319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b3949-144b-4745-863e-1d9b8a559027 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.659994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 780/1574] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:36.661527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6831ef2-8a6c-4f06-9080-a2638e750eb2 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.731940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b3949-144b-4745-863e-1d9b8a559027 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.737317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a9c3-512e-4e23-95cb-2387b4ce745a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.737497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6831ef2-8a6c-4f06-9080-a2638e750eb2 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.737945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a9c3-512e-4e23-95cb-2387b4ce745a admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.738230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a9c3-512e-4e23-95cb-2387b4ce745a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.738535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a9c3-512e-4e23-95cb-2387b4ce745a admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.739186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87a9c3-512e-4e23-95cb-2387b4ce745a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.741146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87a9c3-512e-4e23-95cb-2387b4ce745a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.745255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa9dfe-088d-4119-86a3-3253757b5895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.746615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa9dfe-088d-4119-86a3-3253757b5895 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.746906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa9dfe-088d-4119-86a3-3253757b5895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.747822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa9dfe-088d-4119-86a3-3253757b5895 admin admin] PATH_INFO: `/v3/projects/fa2ae49d03624e8d81556bd5ea73e09c/users/c0c523d97d4c409fab2e3109ac0f678e/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.772929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa9dfe-088d-4119-86a3-3253757b5895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2ae49d03624e8d81556bd5ea73e09c, user_id=c0c523d97d4c409fab2e3109ac0f678e, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.778054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa9dfe-088d-4119-86a3-3253757b5895 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.824607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 795/1575] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:36.825550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a9c3-512e-4e23-95cb-2387b4ce745a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.855780 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 781/1576] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:36 2022] PUT /identity/v3/projects/fa2ae49d03624e8d81556bd5ea73e09c/users/c0c523d97d4c409fab2e3109ac0f678e/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:36.856812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa9dfe-088d-4119-86a3-3253757b5895 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:36.905358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a9c3-512e-4e23-95cb-2387b4ce745a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.910413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:36.926983 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:36.931217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa9dfe-088d-4119-86a3-3253757b5895 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:36.936349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92a539-6d19-4048-a89d-25c03a9b81e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.937030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92a539-6d19-4048-a89d-25c03a9b81e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.937316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92a539-6d19-4048-a89d-25c03a9b81e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.937614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92a539-6d19-4048-a89d-25c03a9b81e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.938250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c92a539-6d19-4048-a89d-25c03a9b81e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.940294 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c92a539-6d19-4048-a89d-25c03a9b81e0 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:36.974375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372d080-8487-4f10-b5d4-10aff8c26dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:36.975125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372d080-8487-4f10-b5d4-10aff8c26dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:36.975450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372d080-8487-4f10-b5d4-10aff8c26dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:36.975728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372d080-8487-4f10-b5d4-10aff8c26dd6 admin admin] PATH_INFO: `/v3/projects/a90db67f8f74446b8cbd913a2a17c0b1/users/1f628503bfd24338b4a351d05a18f126/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:36.992544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372d080-8487-4f10-b5d4-10aff8c26dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a90db67f8f74446b8cbd913a2a17c0b1, user_id=1f628503bfd24338b4a351d05a18f126, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:36.999819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WARNING py.warnings [None req-d372d080-8487-4f10-b5d4-10aff8c26dd6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:29:36.999819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: warnings.warn(msg) Sep 23 01:29:36.999819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: Sep 23 01:29:36.999819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372d080-8487-4f10-b5d4-10aff8c26dd6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.029725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 782/1577] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:36 2022] POST /identity/v3/users => generated 283 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:37.030586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92a539-6d19-4048-a89d-25c03a9b81e0 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.041576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92a539-6d19-4048-a89d-25c03a9b81e0 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.047194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcef92-589a-4392-8d46-1d408c0396d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.050773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcef92-589a-4392-8d46-1d408c0396d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.050773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcef92-589a-4392-8d46-1d408c0396d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.050773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcef92-589a-4392-8d46-1d408c0396d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.050773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcef92-589a-4392-8d46-1d408c0396d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.051210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcef92-589a-4392-8d46-1d408c0396d8 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.097225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 796/1578] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:36 2022] PUT /identity/v3/projects/a90db67f8f74446b8cbd913a2a17c0b1/users/1f628503bfd24338b4a351d05a18f126/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:37.098084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372d080-8487-4f10-b5d4-10aff8c26dd6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.173838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372d080-8487-4f10-b5d4-10aff8c26dd6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.178797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2593f-fd74-4525-bf6b-e0878aaea8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.179018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 783/1579] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:37 2022] POST /identity/v3/projects => generated 393 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:37.179323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2593f-fd74-4525-bf6b-e0878aaea8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.179731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2593f-fd74-4525-bf6b-e0878aaea8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.179854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcef92-589a-4392-8d46-1d408c0396d8 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.180018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2593f-fd74-4525-bf6b-e0878aaea8b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.180807 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2593f-fd74-4525-bf6b-e0878aaea8b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.182799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2593f-fd74-4525-bf6b-e0878aaea8b4 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.248850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcef92-589a-4392-8d46-1d408c0396d8 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.254093 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c84f76-6d40-4131-aa28-79e29bd532d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.254708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c84f76-6d40-4131-aa28-79e29bd532d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.254988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c84f76-6d40-4131-aa28-79e29bd532d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.255282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c84f76-6d40-4131-aa28-79e29bd532d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.255958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c84f76-6d40-4131-aa28-79e29bd532d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.257887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c84f76-6d40-4131-aa28-79e29bd532d4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.293032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 797/1580] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:37 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:37.293995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2593f-fd74-4525-bf6b-e0878aaea8b4 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.360114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2593f-fd74-4525-bf6b-e0878aaea8b4 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.361225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 784/1581] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:37 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:37.362096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c84f76-6d40-4131-aa28-79e29bd532d4 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.365403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5478ad-1959-4156-ba25-702dbf99df89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.366127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5478ad-1959-4156-ba25-702dbf99df89 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.371403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5478ad-1959-4156-ba25-702dbf99df89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.371403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5478ad-1959-4156-ba25-702dbf99df89 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.371403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5478ad-1959-4156-ba25-702dbf99df89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.371403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5478ad-1959-4156-ba25-702dbf99df89 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.373305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c84f76-6d40-4131-aa28-79e29bd532d4 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.378259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9521210-51a1-4550-b9e6-b822176f40eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.378873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9521210-51a1-4550-b9e6-b822176f40eb admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.379159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9521210-51a1-4550-b9e6-b822176f40eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.379434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9521210-51a1-4550-b9e6-b822176f40eb admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.380117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9521210-51a1-4550-b9e6-b822176f40eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.382260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9521210-51a1-4550-b9e6-b822176f40eb admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.470460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 798/1582] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:37 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:37.471954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccac2c-e1c1-4f5e-a3be-060425b939c8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.472315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccac2c-e1c1-4f5e-a3be-060425b939c8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.472591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccac2c-e1c1-4f5e-a3be-060425b939c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.481036 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:37.497758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 785/1583] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:37 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:37.501538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9521210-51a1-4550-b9e6-b822176f40eb admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.505758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-2cccac2c-e1c1-4f5e-a3be-060425b939c8 None None] MFA Rules not processed for user `c0c523d97d4c409fab2e3109ac0f678e`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:37.507586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2cccac2c-e1c1-4f5e-a3be-060425b939c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:37.511133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9521210-51a1-4550-b9e6-b822176f40eb admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.518182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21beab61-7fd1-4967-a828-9bfc895cde6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.519533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beab61-7fd1-4967-a828-9bfc895cde6d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.520019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beab61-7fd1-4967-a828-9bfc895cde6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.520556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beab61-7fd1-4967-a828-9bfc895cde6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.521555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21beab61-7fd1-4967-a828-9bfc895cde6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.524416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21beab61-7fd1-4967-a828-9bfc895cde6d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.546792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 786/1584] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:37.546792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2bff0-332b-4e60-b095-bbd547dc3f67 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.546792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2bff0-332b-4e60-b095-bbd547dc3f67 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.546792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2bff0-332b-4e60-b095-bbd547dc3f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.566510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-fac2bff0-332b-4e60-b095-bbd547dc3f67 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:37.567894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fac2bff0-332b-4e60-b095-bbd547dc3f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:37.575780 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 787/1585] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:37.576580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2bff0-332b-4e60-b095-bbd547dc3f67 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.587468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2bff0-332b-4e60-b095-bbd547dc3f67 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.593347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3aec4-0f72-4aae-a15d-a2bf3bfdd3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.593964 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3aec4-0f72-4aae-a15d-a2bf3bfdd3fc admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.594247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3aec4-0f72-4aae-a15d-a2bf3bfdd3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.594524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3aec4-0f72-4aae-a15d-a2bf3bfdd3fc admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.595169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3aec4-0f72-4aae-a15d-a2bf3bfdd3fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.597180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3aec4-0f72-4aae-a15d-a2bf3bfdd3fc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.598352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 799/1586] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:37 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:37.599181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccac2c-e1c1-4f5e-a3be-060425b939c8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.612973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccac2c-e1c1-4f5e-a3be-060425b939c8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.617942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afcf74-cbe0-43a0-9e9a-623fa1816329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.627731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afcf74-cbe0-43a0-9e9a-623fa1816329 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.627731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afcf74-cbe0-43a0-9e9a-623fa1816329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.627731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afcf74-cbe0-43a0-9e9a-623fa1816329 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.627731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afcf74-cbe0-43a0-9e9a-623fa1816329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.627731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afcf74-cbe0-43a0-9e9a-623fa1816329 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.712606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 788/1587] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:37 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:37.713797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3aec4-0f72-4aae-a15d-a2bf3bfdd3fc admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.724572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3aec4-0f72-4aae-a15d-a2bf3bfdd3fc admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.729080 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30b25c-eda4-4d3b-8a3b-5a6a8b72c969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.729695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30b25c-eda4-4d3b-8a3b-5a6a8b72c969 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.729993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30b25c-eda4-4d3b-8a3b-5a6a8b72c969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.730267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30b25c-eda4-4d3b-8a3b-5a6a8b72c969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.730912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30b25c-eda4-4d3b-8a3b-5a6a8b72c969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.732542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e30b25c-eda4-4d3b-8a3b-5a6a8b72c969 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.743723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 789/1588] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:37.744601 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30b25c-eda4-4d3b-8a3b-5a6a8b72c969 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.753721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 800/1589] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:37 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:37.754560 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afcf74-cbe0-43a0-9e9a-623fa1816329 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.754962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30b25c-eda4-4d3b-8a3b-5a6a8b72c969 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.759950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83529e0-80a4-4cc1-851e-7b8e11d7a379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.771985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83529e0-80a4-4cc1-851e-7b8e11d7a379 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.771985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83529e0-80a4-4cc1-851e-7b8e11d7a379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.771985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83529e0-80a4-4cc1-851e-7b8e11d7a379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.771985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83529e0-80a4-4cc1-851e-7b8e11d7a379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.771985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83529e0-80a4-4cc1-851e-7b8e11d7a379 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.772451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afcf74-cbe0-43a0-9e9a-623fa1816329 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.772451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cb3c5-dc4e-41f9-b33d-412230448fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.772451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cb3c5-dc4e-41f9-b33d-412230448fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.772451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cb3c5-dc4e-41f9-b33d-412230448fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.772745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cb3c5-dc4e-41f9-b33d-412230448fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.772745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cb3c5-dc4e-41f9-b33d-412230448fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.773076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cb3c5-dc4e-41f9-b33d-412230448fcf admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.775103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 790/1590] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:37.776002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83529e0-80a4-4cc1-851e-7b8e11d7a379 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.787929 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 801/1591] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:37.789154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83529e0-80a4-4cc1-851e-7b8e11d7a379 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.789337 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cb3c5-dc4e-41f9-b33d-412230448fcf admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.794360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3504b-7874-498c-a9fd-985db1e8a966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.795326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3504b-7874-498c-a9fd-985db1e8a966 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.795326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3504b-7874-498c-a9fd-985db1e8a966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.795528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3504b-7874-498c-a9fd-985db1e8a966 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.796194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3504b-7874-498c-a9fd-985db1e8a966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.797873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3504b-7874-498c-a9fd-985db1e8a966 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.811211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 791/1592] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:37.811211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3504b-7874-498c-a9fd-985db1e8a966 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.820222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3504b-7874-498c-a9fd-985db1e8a966 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.825790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2231646-6a8c-47e8-8a51-fa67d7d35ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.826422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2231646-6a8c-47e8-8a51-fa67d7d35ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.826700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2231646-6a8c-47e8-8a51-fa67d7d35ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.826976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2231646-6a8c-47e8-8a51-fa67d7d35ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.828280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2231646-6a8c-47e8-8a51-fa67d7d35ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.830322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2231646-6a8c-47e8-8a51-fa67d7d35ee3 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.851146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cb3c5-dc4e-41f9-b33d-412230448fcf admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.854979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3c4e2-b15b-43c4-81b4-ec6c8527bf5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.855629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3c4e2-b15b-43c4-81b4-ec6c8527bf5f service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.857790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3c4e2-b15b-43c4-81b4-ec6c8527bf5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.857790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3c4e2-b15b-43c4-81b4-ec6c8527bf5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.868459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3c4e2-b15b-43c4-81b4-ec6c8527bf5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.870085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3c4e2-b15b-43c4-81b4-ec6c8527bf5f service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.881317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 802/1593] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:37.884228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3c4e2-b15b-43c4-81b4-ec6c8527bf5f service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.892183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3c4e2-b15b-43c4-81b4-ec6c8527bf5f service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.897012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76a3cd-e3d0-4536-8aad-71511fbfe2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.897636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76a3cd-e3d0-4536-8aad-71511fbfe2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.897934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76a3cd-e3d0-4536-8aad-71511fbfe2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.898212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76a3cd-e3d0-4536-8aad-71511fbfe2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.898871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76a3cd-e3d0-4536-8aad-71511fbfe2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.900630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76a3cd-e3d0-4536-8aad-71511fbfe2f1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.911360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 803/1594] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:37.912466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76a3cd-e3d0-4536-8aad-71511fbfe2f1 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.923306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76a3cd-e3d0-4536-8aad-71511fbfe2f1 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:37.928854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1489b-cbd5-4942-b685-5572d8dab809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:37.929756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1489b-cbd5-4942-b685-5572d8dab809 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:37.930122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1489b-cbd5-4942-b685-5572d8dab809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:37.930322 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1489b-cbd5-4942-b685-5572d8dab809 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:37.931051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1489b-cbd5-4942-b685-5572d8dab809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:37.932730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1489b-cbd5-4942-b685-5572d8dab809 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:37.941890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 792/1595] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:37 2022] POST /identity/v3/projects => generated 405 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:37.941890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2231646-6a8c-47e8-8a51-fa67d7d35ee3 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.944755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:37.950140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 804/1596] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:37.950140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1489b-cbd5-4942-b685-5572d8dab809 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:37.958350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1489b-cbd5-4942-b685-5572d8dab809 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.008142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2231646-6a8c-47e8-8a51-fa67d7d35ee3 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.009170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:38.013231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc433c90-84f4-44a1-aa71-6b2ce88146af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.013866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc433c90-84f4-44a1-aa71-6b2ce88146af admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.014143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc433c90-84f4-44a1-aa71-6b2ce88146af admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.014419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc433c90-84f4-44a1-aa71-6b2ce88146af admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.015141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc433c90-84f4-44a1-aa71-6b2ce88146af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.016728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc433c90-84f4-44a1-aa71-6b2ce88146af admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.025432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69edae-5f5b-4f58-8724-565d7a2c1567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.026181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69edae-5f5b-4f58-8724-565d7a2c1567 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.029791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69edae-5f5b-4f58-8724-565d7a2c1567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.029791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69edae-5f5b-4f58-8724-565d7a2c1567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.029791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69edae-5f5b-4f58-8724-565d7a2c1567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.029791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69edae-5f5b-4f58-8724-565d7a2c1567 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.031881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 793/1597] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:38.033038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc433c90-84f4-44a1-aa71-6b2ce88146af admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.045706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 805/1598] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:38.045706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69edae-5f5b-4f58-8724-565d7a2c1567 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.046040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc433c90-84f4-44a1-aa71-6b2ce88146af admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.051251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ae1b7-4987-4254-8124-2bb8fb1773af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.051913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ae1b7-4987-4254-8124-2bb8fb1773af admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.056983 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ae1b7-4987-4254-8124-2bb8fb1773af admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.056983 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ae1b7-4987-4254-8124-2bb8fb1773af admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.056983 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ae1b7-4987-4254-8124-2bb8fb1773af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.056983 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ae1b7-4987-4254-8124-2bb8fb1773af admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.057565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69edae-5f5b-4f58-8724-565d7a2c1567 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.063977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d1f86-0d46-48d4-ace2-d85c0c59e206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.064774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d1f86-0d46-48d4-ace2-d85c0c59e206 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.065111 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d1f86-0d46-48d4-ace2-d85c0c59e206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.065451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d1f86-0d46-48d4-ace2-d85c0c59e206 admin admin] PATH_INFO: `/v3/projects/eb91993f6c5445abb3780710cc9dba24/users/a26346680d984c6e8dbb67da0b471ce7/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.067482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 794/1599] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:38.068321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ae1b7-4987-4254-8124-2bb8fb1773af admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.079300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ae1b7-4987-4254-8124-2bb8fb1773af admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.091235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d1f86-0d46-48d4-ace2-d85c0c59e206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb91993f6c5445abb3780710cc9dba24, user_id=a26346680d984c6e8dbb67da0b471ce7, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.091531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262b8f8-9308-4dee-8971-7ee08f5e95ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.092171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b8f8-9308-4dee-8971-7ee08f5e95ec admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.092449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b8f8-9308-4dee-8971-7ee08f5e95ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.092783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b8f8-9308-4dee-8971-7ee08f5e95ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.094246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262b8f8-9308-4dee-8971-7ee08f5e95ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.095582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0d1f86-0d46-48d4-ace2-d85c0c59e206 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.097237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262b8f8-9308-4dee-8971-7ee08f5e95ec admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.108302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 795/1600] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:38.109302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262b8f8-9308-4dee-8971-7ee08f5e95ec admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.119309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262b8f8-9308-4dee-8971-7ee08f5e95ec admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.124322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8ccd7-491c-4e7d-a49b-41d297b88007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.124971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8ccd7-491c-4e7d-a49b-41d297b88007 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.125262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8ccd7-491c-4e7d-a49b-41d297b88007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.125542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8ccd7-491c-4e7d-a49b-41d297b88007 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.126193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8ccd7-491c-4e7d-a49b-41d297b88007 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.128222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab8ccd7-491c-4e7d-a49b-41d297b88007 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.173808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 806/1601] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:38 2022] PUT /identity/v3/projects/eb91993f6c5445abb3780710cc9dba24/users/a26346680d984c6e8dbb67da0b471ce7/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:38.175512 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d1f86-0d46-48d4-ace2-d85c0c59e206 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.223423 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 796/1602] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:38 2022] POST /identity/v3/users => generated 309 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:38.224776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8ccd7-491c-4e7d-a49b-41d297b88007 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.237196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8ccd7-491c-4e7d-a49b-41d297b88007 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.279330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0d1f86-0d46-48d4-ace2-d85c0c59e206 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.285624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdcf22-548f-472c-90b3-6cff714c52b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.286384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdcf22-548f-472c-90b3-6cff714c52b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.286677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdcf22-548f-472c-90b3-6cff714c52b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.286966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdcf22-548f-472c-90b3-6cff714c52b1 admin admin] PATH_INFO: `/v3/projects/92e9b7f9904f437f91148da9fc521326/users/1633ea35eda34ffe884140eaab1ae909/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.306611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdcf22-548f-472c-90b3-6cff714c52b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e9b7f9904f437f91148da9fc521326, user_id=1633ea35eda34ffe884140eaab1ae909, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.309602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cdcf22-548f-472c-90b3-6cff714c52b1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.337215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1231975-5a52-4fdd-b6a4-859ad4d34f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.338268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1231975-5a52-4fdd-b6a4-859ad4d34f24 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.338268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1231975-5a52-4fdd-b6a4-859ad4d34f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.338572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1231975-5a52-4fdd-b6a4-859ad4d34f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.339291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1231975-5a52-4fdd-b6a4-859ad4d34f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.340956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1231975-5a52-4fdd-b6a4-859ad4d34f24 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.351801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 797/1603] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:38.354230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1231975-5a52-4fdd-b6a4-859ad4d34f24 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.362890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1231975-5a52-4fdd-b6a4-859ad4d34f24 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.367522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4bf5c-4115-4c9f-873d-c3d881a20675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.368249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4bf5c-4115-4c9f-873d-c3d881a20675 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.368532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4bf5c-4115-4c9f-873d-c3d881a20675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.368830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4bf5c-4115-4c9f-873d-c3d881a20675 admin admin] PATH_INFO: `/v3/projects/6df4753bdb2e4557afbc2e9434fc12e0/users/c04146f1bd0d4c12b549a3cbf766796d/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.386710 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4bf5c-4115-4c9f-873d-c3d881a20675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df4753bdb2e4557afbc2e9434fc12e0, user_id=c04146f1bd0d4c12b549a3cbf766796d, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.389880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4bf5c-4115-4c9f-873d-c3d881a20675 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.429787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 807/1604] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:38 2022] PUT /identity/v3/projects/92e9b7f9904f437f91148da9fc521326/users/1633ea35eda34ffe884140eaab1ae909/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:38.430630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdcf22-548f-472c-90b3-6cff714c52b1 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.519208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdcf22-548f-472c-90b3-6cff714c52b1 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.522920 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 798/1605] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:38 2022] PUT /identity/v3/projects/6df4753bdb2e4557afbc2e9434fc12e0/users/c04146f1bd0d4c12b549a3cbf766796d/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:38.524247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4bf5c-4115-4c9f-873d-c3d881a20675 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.571182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:38.574613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:38.586603 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326492c5-317d-448a-a806-423ebe4da422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.587291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326492c5-317d-448a-a806-423ebe4da422 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.587723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326492c5-317d-448a-a806-423ebe4da422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.587970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326492c5-317d-448a-a806-423ebe4da422 admin admin] PATH_INFO: `/v3/projects/cc926b4dede94c3b92441b0c610771f6/users/62f175cb5d1e4ac8a0ee95cf7718837b/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.603209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4bf5c-4115-4c9f-873d-c3d881a20675 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.604674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd1375-1e36-4a25-9923-3aadc099a025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.605611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd1375-1e36-4a25-9923-3aadc099a025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.605989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd1375-1e36-4a25-9923-3aadc099a025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.606380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd1375-1e36-4a25-9923-3aadc099a025 admin admin] PATH_INFO: `/v3/projects/1b8939e407ff4bdc81191af7f0593cd5/users/da9978cab3fb4bdda39a2e4718ee2cc7/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.606567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326492c5-317d-448a-a806-423ebe4da422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc926b4dede94c3b92441b0c610771f6, user_id=62f175cb5d1e4ac8a0ee95cf7718837b, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.610575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326492c5-317d-448a-a806-423ebe4da422 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.623812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd1375-1e36-4a25-9923-3aadc099a025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b8939e407ff4bdc81191af7f0593cd5, user_id=da9978cab3fb4bdda39a2e4718ee2cc7, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.627265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd1375-1e36-4a25-9923-3aadc099a025 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.745255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 808/1606] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:38 2022] PUT /identity/v3/projects/cc926b4dede94c3b92441b0c610771f6/users/62f175cb5d1e4ac8a0ee95cf7718837b/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:38.746220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11b724-5d6a-422b-99b4-6499c65b2671 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.746510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11b724-5d6a-422b-99b4-6499c65b2671 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.746789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a11b724-5d6a-422b-99b4-6499c65b2671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.753556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 799/1607] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:38 2022] PUT /identity/v3/projects/1b8939e407ff4bdc81191af7f0593cd5/users/da9978cab3fb4bdda39a2e4718ee2cc7/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:38.753556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd1375-1e36-4a25-9923-3aadc099a025 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.798917 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5a11b724-5d6a-422b-99b4-6499c65b2671 None None] MFA Rules not processed for user `a26346680d984c6e8dbb67da0b471ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:38.801112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5a11b724-5d6a-422b-99b4-6499c65b2671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:38.847618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd1375-1e36-4a25-9923-3aadc099a025 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.853356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951e02d-3659-436a-a4b6-8c7735413634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.853989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951e02d-3659-436a-a4b6-8c7735413634 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.854271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951e02d-3659-436a-a4b6-8c7735413634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.854550 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951e02d-3659-436a-a4b6-8c7735413634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.855260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951e02d-3659-436a-a4b6-8c7735413634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.856944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951e02d-3659-436a-a4b6-8c7735413634 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.868690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 800/1608] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:38.869579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951e02d-3659-436a-a4b6-8c7735413634 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:38.880884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951e02d-3659-436a-a4b6-8c7735413634 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:38.885906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc15ab-3c72-45a4-bf55-ecc373532404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:38.886573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc15ab-3c72-45a4-bf55-ecc373532404 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.886854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc15ab-3c72-45a4-bf55-ecc373532404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.887133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc15ab-3c72-45a4-bf55-ecc373532404 admin admin] PATH_INFO: `/v3/projects/c44d377f4f9749798e0a1c4ee9a6a5ee/users/0cdb50891eb44beba40f2bd9dbeb4759/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.895215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 809/1609] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:38.897125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6752d8-6a63-4d73-9c10-91b7a9d76038 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.897518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6752d8-6a63-4d73-9c10-91b7a9d76038 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.897923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6752d8-6a63-4d73-9c10-91b7a9d76038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.910851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc15ab-3c72-45a4-bf55-ecc373532404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c44d377f4f9749798e0a1c4ee9a6a5ee, user_id=0cdb50891eb44beba40f2bd9dbeb4759, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:38.913676 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc15ab-3c72-45a4-bf55-ecc373532404 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:38.920131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ac6752d8-6a63-4d73-9c10-91b7a9d76038 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:38.921957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ac6752d8-6a63-4d73-9c10-91b7a9d76038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:38.930140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 810/1610] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:38.931477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cd199-bc6d-40a8-8c46-61b76d2f137f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.931805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cd199-bc6d-40a8-8c46-61b76d2f137f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.932088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68cd199-bc6d-40a8-8c46-61b76d2f137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:38.961667 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a68cd199-bc6d-40a8-8c46-61b76d2f137f None None] MFA Rules not processed for user `c04146f1bd0d4c12b549a3cbf766796d`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:38.962888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a68cd199-bc6d-40a8-8c46-61b76d2f137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:38.977430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 801/1611] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:38 2022] PUT /identity/v3/projects/c44d377f4f9749798e0a1c4ee9a6a5ee/users/0cdb50891eb44beba40f2bd9dbeb4759/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:38.978714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392a76c-73c8-4693-abfa-62c671b048fa None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:38.979010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392a76c-73c8-4693-abfa-62c671b048fa None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:38.979273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392a76c-73c8-4693-abfa-62c671b048fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.003068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f392a76c-73c8-4693-abfa-62c671b048fa None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:39.004291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f392a76c-73c8-4693-abfa-62c671b048fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:39.066155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 811/1612] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:39.068163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3436a-fd2b-49e4-83a9-3128b86d397e None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.068554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3436a-fd2b-49e4-83a9-3128b86d397e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.068932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3436a-fd2b-49e4-83a9-3128b86d397e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.078020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 802/1613] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:39.079406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392a76c-73c8-4693-abfa-62c671b048fa None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:39.091456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392a76c-73c8-4693-abfa-62c671b048fa None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:39.095470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3cb3436a-fd2b-49e4-83a9-3128b86d397e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:39.096955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3cb3436a-fd2b-49e4-83a9-3128b86d397e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:39.098985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688bc7e9-a490-4ffc-b5a4-ffb1f0573964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:39.099813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bc7e9-a490-4ffc-b5a4-ffb1f0573964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.100092 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bc7e9-a490-4ffc-b5a4-ffb1f0573964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.100515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688bc7e9-a490-4ffc-b5a4-ffb1f0573964 admin admin] PATH_INFO: `/v3/projects/156eb6f806034087a60316940f2286a0/users/f0d56189d2404ec88319445b4eb5e9d8/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.105300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 812/1614] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:39.106187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3436a-fd2b-49e4-83a9-3128b86d397e None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:39.119908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688bc7e9-a490-4ffc-b5a4-ffb1f0573964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156eb6f806034087a60316940f2286a0, user_id=f0d56189d2404ec88319445b4eb5e9d8, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:39.123032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688bc7e9-a490-4ffc-b5a4-ffb1f0573964 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:39.155478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:39.170989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3436a-fd2b-49e4-83a9-3128b86d397e None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:39.175725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb96da9-6e8b-4808-9912-1972421b3a80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:39.176355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb96da9-6e8b-4808-9912-1972421b3a80 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.176637 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb96da9-6e8b-4808-9912-1972421b3a80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.176915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb96da9-6e8b-4808-9912-1972421b3a80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.198971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 803/1615] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:39 2022] PUT /identity/v3/projects/156eb6f806034087a60316940f2286a0/users/f0d56189d2404ec88319445b4eb5e9d8/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:39.200273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688bc7e9-a490-4ffc-b5a4-ffb1f0573964 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:39.202382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-688bc7e9-a490-4ffc-b5a4-ffb1f0573964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:39.214414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:39.241486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb96da9-6e8b-4808-9912-1972421b3a80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:39.242943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb96da9-6e8b-4808-9912-1972421b3a80 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:39.252870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 813/1616] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:39.254658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a737b-cab6-4e6d-aed9-5b472949f5fb None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.255072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a737b-cab6-4e6d-aed9-5b472949f5fb None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.255447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a737b-cab6-4e6d-aed9-5b472949f5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.288730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688bc7e9-a490-4ffc-b5a4-ffb1f0573964 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:39.579786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-833a737b-cab6-4e6d-aed9-5b472949f5fb None None] MFA Rules not processed for user `0cdb50891eb44beba40f2bd9dbeb4759`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:39.579786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-833a737b-cab6-4e6d-aed9-5b472949f5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:39.581087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a4be0-69b7-4dd0-81b6-4544ca062c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:39.581087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a4be0-69b7-4dd0-81b6-4544ca062c55 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.581087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a4be0-69b7-4dd0-81b6-4544ca062c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.581087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a4be0-69b7-4dd0-81b6-4544ca062c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.582181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a4be0-69b7-4dd0-81b6-4544ca062c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:39.582181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a4be0-69b7-4dd0-81b6-4544ca062c55 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:39.582181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 804/1617] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:39.582181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a4be0-69b7-4dd0-81b6-4544ca062c55 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:39.582181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a4be0-69b7-4dd0-81b6-4544ca062c55 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:39.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4303923-a711-4419-8812-aec91cba3dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:39.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303923-a711-4419-8812-aec91cba3dba admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303923-a711-4419-8812-aec91cba3dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.582773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4303923-a711-4419-8812-aec91cba3dba admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.583444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4303923-a711-4419-8812-aec91cba3dba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:39.583444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4303923-a711-4419-8812-aec91cba3dba admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:39.583588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 814/1618] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:39.583588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a737b-cab6-4e6d-aed9-5b472949f5fb None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:39.583588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a737b-cab6-4e6d-aed9-5b472949f5fb None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:39.583588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e8c91-7ed5-4230-9520-eb3b24322cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:39.583922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e8c91-7ed5-4230-9520-eb3b24322cea service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.583922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e8c91-7ed5-4230-9520-eb3b24322cea service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.583922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e8c91-7ed5-4230-9520-eb3b24322cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.583922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e8c91-7ed5-4230-9520-eb3b24322cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:39.583922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e8c91-7ed5-4230-9520-eb3b24322cea service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:39.583922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 815/1619] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:39.584417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e8c91-7ed5-4230-9520-eb3b24322cea service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:39.584417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e8c91-7ed5-4230-9520-eb3b24322cea service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:39.585679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56168bcc-0027-465f-a75f-59cc1428a445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:39.586626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56168bcc-0027-465f-a75f-59cc1428a445 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.587057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56168bcc-0027-465f-a75f-59cc1428a445 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.587510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56168bcc-0027-465f-a75f-59cc1428a445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.600160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56168bcc-0027-465f-a75f-59cc1428a445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:39.601940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56168bcc-0027-465f-a75f-59cc1428a445 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:39.614552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 816/1620] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:39.615889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56168bcc-0027-465f-a75f-59cc1428a445 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:39.627389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56168bcc-0027-465f-a75f-59cc1428a445 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:39.633135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e38c6-4dc0-46be-9070-2df37244cbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:39.634028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e38c6-4dc0-46be-9070-2df37244cbf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.634519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e38c6-4dc0-46be-9070-2df37244cbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.635120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e38c6-4dc0-46be-9070-2df37244cbf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.638035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e38c6-4dc0-46be-9070-2df37244cbf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:39.641708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e38c6-4dc0-46be-9070-2df37244cbf1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:39.674034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 817/1621] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:39.677121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c83b3-c2f8-416f-a258-5a55f47a10e4 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.688970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c83b3-c2f8-416f-a258-5a55f47a10e4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.688970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c83b3-c2f8-416f-a258-5a55f47a10e4 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.688970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 818/1622] 10.222.0.64 () {54 vars in 961 bytes} [Fri Sep 23 01:29:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:29:39.688970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c83b3-c2f8-416f-a258-5a55f47a10e4 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:39.704223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c83b3-c2f8-416f-a258-5a55f47a10e4 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:39.711793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d21fe-b429-48f0-9728-a0ca7dad1da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:39.712884 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d21fe-b429-48f0-9728-a0ca7dad1da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.713409 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d21fe-b429-48f0-9728-a0ca7dad1da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.714108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d21fe-b429-48f0-9728-a0ca7dad1da7 admin admin] PATH_INFO: `/v3/projects/156eb6f806034087a60316940f2286a0/users/f0d56189d2404ec88319445b4eb5e9d8/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.716331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d21fe-b429-48f0-9728-a0ca7dad1da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156eb6f806034087a60316940f2286a0, user_id=f0d56189d2404ec88319445b4eb5e9d8, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:39.719954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d21fe-b429-48f0-9728-a0ca7dad1da7 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:39.735745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 819/1623] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:39 2022] PUT /identity/v3/projects/156eb6f806034087a60316940f2286a0/users/f0d56189d2404ec88319445b4eb5e9d8/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:39.737410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823257cb-7235-465c-903a-287c71870690 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.737715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823257cb-7235-465c-903a-287c71870690 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.737994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823257cb-7235-465c-903a-287c71870690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.759638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 805/1624] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:39 2022] POST /identity/v3/projects => generated 373 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:39.759638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30d577-d8bc-4e54-9dd8-05abcab8de98 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.759638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30d577-d8bc-4e54-9dd8-05abcab8de98 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.759638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30d577-d8bc-4e54-9dd8-05abcab8de98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:39.780532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-823257cb-7235-465c-903a-287c71870690 None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:39.782284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-823257cb-7235-465c-903a-287c71870690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:39.789972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-cd30d577-d8bc-4e54-9dd8-05abcab8de98 None None] MFA Rules not processed for user `f0d56189d2404ec88319445b4eb5e9d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:39.791222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cd30d577-d8bc-4e54-9dd8-05abcab8de98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:39.795604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:39.877764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 820/1625] 10.222.0.64 () {58 vars in 1058 bytes} [Fri Sep 23 01:29:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:39.879956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823257cb-7235-465c-903a-287c71870690 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:39.915831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 806/1626] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:39 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:39.918221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30d577-d8bc-4e54-9dd8-05abcab8de98 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:39.952934 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30d577-d8bc-4e54-9dd8-05abcab8de98 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:39.975875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed89bf-bffe-4ea8-a92d-d855d6d4ad05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:39.976944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed89bf-bffe-4ea8-a92d-d855d6d4ad05 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:39.976944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed89bf-bffe-4ea8-a92d-d855d6d4ad05 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:39.977376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed89bf-bffe-4ea8-a92d-d855d6d4ad05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:40.015076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:40.048584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823257cb-7235-465c-903a-287c71870690 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:40.057606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9258b-b10b-4153-8584-dd9ccfc79193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:40.058254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9258b-b10b-4153-8584-dd9ccfc79193 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:40.058533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9258b-b10b-4153-8584-dd9ccfc79193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:40.058803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9258b-b10b-4153-8584-dd9ccfc79193 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:40.060242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9258b-b10b-4153-8584-dd9ccfc79193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:40.062946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9258b-b10b-4153-8584-dd9ccfc79193 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:40.125569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed89bf-bffe-4ea8-a92d-d855d6d4ad05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:40.129917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed89bf-bffe-4ea8-a92d-d855d6d4ad05 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:40.148271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 807/1627] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:40.149524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed89bf-bffe-4ea8-a92d-d855d6d4ad05 service nova] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:40.217241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 821/1628] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:39 2022] POST /identity/v3/users => generated 293 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:40.222037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9258b-b10b-4153-8584-dd9ccfc79193 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:40.248733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9258b-b10b-4153-8584-dd9ccfc79193 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:40.256969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed89bf-bffe-4ea8-a92d-d855d6d4ad05 service nova] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:40.262816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888ce6b-3dc3-4928-9746-10ea9f581750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:40.263912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888ce6b-3dc3-4928-9746-10ea9f581750 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:40.264202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888ce6b-3dc3-4928-9746-10ea9f581750 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:40.269157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888ce6b-3dc3-4928-9746-10ea9f581750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:40.271282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9f24e-5acf-42e2-a5de-efe99f7e5d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:40.275600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9f24e-5acf-42e2-a5de-efe99f7e5d64 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:40.276421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9f24e-5acf-42e2-a5de-efe99f7e5d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:40.276778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9f24e-5acf-42e2-a5de-efe99f7e5d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:40.278885 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9f24e-5acf-42e2-a5de-efe99f7e5d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:40.280973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9f24e-5acf-42e2-a5de-efe99f7e5d64 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:40.281382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888ce6b-3dc3-4928-9746-10ea9f581750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:40.283250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888ce6b-3dc3-4928-9746-10ea9f581750 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:40.304173 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 808/1629] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:40 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:40.314206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 822/1630] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:40.317455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888ce6b-3dc3-4928-9746-10ea9f581750 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:40.331430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888ce6b-3dc3-4928-9746-10ea9f581750 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:40.338568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732620ad-a5b5-4dfe-9458-dd14e46c3f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:40.339707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732620ad-a5b5-4dfe-9458-dd14e46c3f0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:40.340692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732620ad-a5b5-4dfe-9458-dd14e46c3f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:40.341128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732620ad-a5b5-4dfe-9458-dd14e46c3f0c admin admin] PATH_INFO: `/v3/projects/f10e431256454066a2e24f6bb917ac89/users/47c320152ed047bb8b3ac345de78062c/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:40.370830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732620ad-a5b5-4dfe-9458-dd14e46c3f0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10e431256454066a2e24f6bb917ac89, user_id=47c320152ed047bb8b3ac345de78062c, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:40.376069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732620ad-a5b5-4dfe-9458-dd14e46c3f0c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:40.529332 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 809/1631] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:40 2022] PUT /identity/v3/projects/f10e431256454066a2e24f6bb917ac89/users/47c320152ed047bb8b3ac345de78062c/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:40.546124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9f24e-5acf-42e2-a5de-efe99f7e5d64 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:40.558113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9f24e-5acf-42e2-a5de-efe99f7e5d64 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:40.574401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41055fc7-dce1-443c-aa28-aab4933b5316 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:40.575992 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41055fc7-dce1-443c-aa28-aab4933b5316 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:40.576705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41055fc7-dce1-443c-aa28-aab4933b5316 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:40.579065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 810/1632] 10.222.0.64 () {54 vars in 912 bytes} [Fri Sep 23 01:29:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:29:40.580014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41055fc7-dce1-443c-aa28-aab4933b5316 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:40.741162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca354dba-9088-43ad-bd57-b545056246d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:40.743421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca354dba-9088-43ad-bd57-b545056246d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:40.744336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca354dba-9088-43ad-bd57-b545056246d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:40.744624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca354dba-9088-43ad-bd57-b545056246d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:40.746245 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca354dba-9088-43ad-bd57-b545056246d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:40.750665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca354dba-9088-43ad-bd57-b545056246d6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:40.770098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41055fc7-dce1-443c-aa28-aab4933b5316 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:40.784074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 823/1633] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:40.785114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:40.788438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb27d3a-1cff-4a24-b6b7-43e8defe5c94 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:40.788947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb27d3a-1cff-4a24-b6b7-43e8defe5c94 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:40.789106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb27d3a-1cff-4a24-b6b7-43e8defe5c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:40.793711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:40.794225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:40.794782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] PATH_INFO: `/v3/users/c0c523d97d4c409fab2e3109ac0f678e` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:40.797756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0c523d97d4c409fab2e3109ac0f678e)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:40.800932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:40.853685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ebb27d3a-1cff-4a24-b6b7-43e8defe5c94 None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:40.856972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ebb27d3a-1cff-4a24-b6b7-43e8defe5c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:40.995408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:41.019399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 824/1634] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:29:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:41.020334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb27d3a-1cff-4a24-b6b7-43e8defe5c94 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:41.033432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb27d3a-1cff-4a24-b6b7-43e8defe5c94 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:41.076156 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4adad7-f71b-464a-8880-af9985fb0d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:41.078022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4adad7-f71b-464a-8880-af9985fb0d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:41.084621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4adad7-f71b-464a-8880-af9985fb0d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:41.084883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4adad7-f71b-464a-8880-af9985fb0d2e admin admin] PATH_INFO: `/v3/projects/f10e431256454066a2e24f6bb917ac89/users/47c320152ed047bb8b3ac345de78062c/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:41.093202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4adad7-f71b-464a-8880-af9985fb0d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10e431256454066a2e24f6bb917ac89, user_id=47c320152ed047bb8b3ac345de78062c, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:41.101024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4adad7-f71b-464a-8880-af9985fb0d2e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:41.139599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 825/1635] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:41 2022] PUT /identity/v3/projects/f10e431256454066a2e24f6bb917ac89/users/47c320152ed047bb8b3ac345de78062c/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:41.139599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4adad7-f71b-464a-8880-af9985fb0d2e admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:41.223331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0c523d97d4c409fab2e3109ac0f678e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:41.289222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4adad7-f71b-464a-8880-af9985fb0d2e admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:41.300068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb7fbe-0582-4dad-96b2-60527a618838 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:41.300758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb7fbe-0582-4dad-96b2-60527a618838 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:41.301059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb7fbe-0582-4dad-96b2-60527a618838 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:41.301475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb7fbe-0582-4dad-96b2-60527a618838 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:41.349417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0c523d97d4c409fab2e3109ac0f678e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:41.350753 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0c523d97d4c409fab2e3109ac0f678e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:41.372285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0c523d97d4c409fab2e3109ac0f678e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:41.373207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2aa71e39-824f-4a20-a601-41fcbf9e1fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0c523d97d4c409fab2e3109ac0f678e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:41.395917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 811/1636] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:29:40 2022] DELETE /identity/v3/users/c0c523d97d4c409fab2e3109ac0f678e => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:41.395917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd648e24-4259-43fd-9912-992460c9bf4c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:41.395917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd648e24-4259-43fd-9912-992460c9bf4c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:41.395917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd648e24-4259-43fd-9912-992460c9bf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:41.414528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb7fbe-0582-4dad-96b2-60527a618838 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:41.416637 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb7fbe-0582-4dad-96b2-60527a618838 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:41.436389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 826/1637] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:41.440620 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681755b-ff0b-4ce5-8245-fb9d5c609635 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:41.441121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681755b-ff0b-4ce5-8245-fb9d5c609635 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:41.441940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681755b-ff0b-4ce5-8245-fb9d5c609635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:41.484374 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-8681755b-ff0b-4ce5-8245-fb9d5c609635 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:41.485167 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-cd648e24-4259-43fd-9912-992460c9bf4c None None] MFA Rules not processed for user `47c320152ed047bb8b3ac345de78062c`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:41.490955 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:41.492490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cd648e24-4259-43fd-9912-992460c9bf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:41.495163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-8681755b-ff0b-4ce5-8245-fb9d5c609635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:41.610142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 827/1638] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:41.622195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681755b-ff0b-4ce5-8245-fb9d5c609635 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:41.624401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-8681755b-ff0b-4ce5-8245-fb9d5c609635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:41.632384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 812/1639] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:41.646433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd648e24-4259-43fd-9912-992460c9bf4c None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:41.649100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cd648e24-4259-43fd-9912-992460c9bf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:41.750604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681755b-ff0b-4ce5-8245-fb9d5c609635 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:41.755015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa912dc2-2037-4ee7-be97-fcd3ea82efdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:41.755686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa912dc2-2037-4ee7-be97-fcd3ea82efdb service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:41.756024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa912dc2-2037-4ee7-be97-fcd3ea82efdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:41.756304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa912dc2-2037-4ee7-be97-fcd3ea82efdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:41.766575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd648e24-4259-43fd-9912-992460c9bf4c None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:41.770400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa912dc2-2037-4ee7-be97-fcd3ea82efdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:41.770866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a77fa-4fdc-44e7-9440-1cd8ada10816 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:41.771512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a77fa-4fdc-44e7-9440-1cd8ada10816 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:41.772142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a77fa-4fdc-44e7-9440-1cd8ada10816 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:41.772481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a77fa-4fdc-44e7-9440-1cd8ada10816 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:41.775268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa912dc2-2037-4ee7-be97-fcd3ea82efdb service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:41.787292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a77fa-4fdc-44e7-9440-1cd8ada10816 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:41.788921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a77fa-4fdc-44e7-9440-1cd8ada10816 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:41.789574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 828/1640] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:29:41.801725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 813/1641] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:29:42.503460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:42.883911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0186-d6f3-4e08-a8de-0db2adf1e8e7 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:42.884547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0186-d6f3-4e08-a8de-0db2adf1e8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:42.884547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0186-d6f3-4e08-a8de-0db2adf1e8e7 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:42.886692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 829/1642] 10.222.0.64 () {54 vars in 961 bytes} [Fri Sep 23 01:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:29:42.896984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef24e6-78bb-4b33-9a1b-2ca982f4a3a9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:42.897994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef24e6-78bb-4b33-9a1b-2ca982f4a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:42.898494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef24e6-78bb-4b33-9a1b-2ca982f4a3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:42.953353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-75ef24e6-78bb-4b33-9a1b-2ca982f4a3a9 None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:42.955009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-75ef24e6-78bb-4b33-9a1b-2ca982f4a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:43.139596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 814/1643] 10.222.0.64 () {58 vars in 1058 bytes} [Fri Sep 23 01:29:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:43.145702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7db0dd-5b19-4319-a4ef-f9723178f10d None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:43.146202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7db0dd-5b19-4319-a4ef-f9723178f10d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:43.146767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7db0dd-5b19-4319-a4ef-f9723178f10d None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:43.148353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 830/1644] 10.222.0.64 () {54 vars in 912 bytes} [Fri Sep 23 01:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:29:43.157594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209833d6-bc78-4d11-adc2-bbe68a3186c4 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:43.158204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209833d6-bc78-4d11-adc2-bbe68a3186c4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:43.158643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209833d6-bc78-4d11-adc2-bbe68a3186c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:43.186550 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-209833d6-bc78-4d11-adc2-bbe68a3186c4 None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:43.189781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-209833d6-bc78-4d11-adc2-bbe68a3186c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:43.206389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 815/1645] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:29:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:43.216697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7db0dd-5b19-4319-a4ef-f9723178f10d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:43.219093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ab7db0dd-5b19-4319-a4ef-f9723178f10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:43.235138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7db0dd-5b19-4319-a4ef-f9723178f10d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:43.248560 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae84c1-beaf-4bfc-b97c-35f5004e489f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:43.249096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae84c1-beaf-4bfc-b97c-35f5004e489f service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:43.249511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae84c1-beaf-4bfc-b97c-35f5004e489f service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:43.249901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae84c1-beaf-4bfc-b97c-35f5004e489f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:43.271407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae84c1-beaf-4bfc-b97c-35f5004e489f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:43.273988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae84c1-beaf-4bfc-b97c-35f5004e489f service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:43.297756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 831/1646] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:29:43.364835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209833d6-bc78-4d11-adc2-bbe68a3186c4 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:43.366555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-209833d6-bc78-4d11-adc2-bbe68a3186c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:43.402389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209833d6-bc78-4d11-adc2-bbe68a3186c4 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:43.414234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:43.415112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:43.415401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:43.415931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] PATH_INFO: `/v3/projects/fa2ae49d03624e8d81556bd5ea73e09c` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:43.419335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2ae49d03624e8d81556bd5ea73e09c)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:43.421482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: WARNING py.warnings [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:29:43.421482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: warnings.warn(msg) Sep 23 01:29:43.421482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: Sep 23 01:29:43.422756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:43.482150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae84c1-beaf-4bfc-b97c-35f5004e489f service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:43.485393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-feae84c1-beaf-4bfc-b97c-35f5004e489f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:43.514028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae84c1-beaf-4bfc-b97c-35f5004e489f service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:43.526844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:43.527713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:43.529005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:43.530839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] PATH_INFO: `/v3/users/c04146f1bd0d4c12b549a3cbf766796d` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:43.533067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c04146f1bd0d4c12b549a3cbf766796d)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:43.534729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: WARNING py.warnings [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 23 01:29:43.534729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: warnings.warn(msg) Sep 23 01:29:43.534729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: Sep 23 01:29:43.536345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:43.604258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:43.641661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2ae49d03624e8d81556bd5ea73e09c', 'request_id': 'req-773cf347-82dd-4c9a-bad9-8719b51c21c8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:43.648750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2ae49d03624e8d81556bd5ea73e09c', 'request_id': 'req-773cf347-82dd-4c9a-bad9-8719b51c21c8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:43.776764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2ae49d03624e8d81556bd5ea73e09c', 'request_id': 'req-773cf347-82dd-4c9a-bad9-8719b51c21c8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:43.838637 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c04146f1bd0d4c12b549a3cbf766796d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:43.841545 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c04146f1bd0d4c12b549a3cbf766796d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:43.842931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c04146f1bd0d4c12b549a3cbf766796d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:43.877960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c04146f1bd0d4c12b549a3cbf766796d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:43.879281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] The token cache is being invalidate because project fa2ae49d03624e8d81556bd5ea73e09c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:29:43.880027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:43.884877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:43.893238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 816/1647] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:29:43 2022] DELETE /identity/v3/projects/fa2ae49d03624e8d81556bd5ea73e09c => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:43.897353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:43.902814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:43.930953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-eef1acaf-cdc1-498c-81f5-46716c23175c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c04146f1bd0d4c12b549a3cbf766796d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:44.016566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 832/1648] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:29:43 2022] DELETE /identity/v3/users/c04146f1bd0d4c12b549a3cbf766796d => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:44.020939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e4d81-54ac-4b74-b90c-182f00085b04 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:44.021360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e4d81-54ac-4b74-b90c-182f00085b04 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:44.023745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1e4d81-54ac-4b74-b90c-182f00085b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:44.078747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-7a1e4d81-54ac-4b74-b90c-182f00085b04 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:44.080287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7a1e4d81-54ac-4b74-b90c-182f00085b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:44.199125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:44.245481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 833/1649] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:44.294271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e4d81-54ac-4b74-b90c-182f00085b04 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:44.296411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7a1e4d81-54ac-4b74-b90c-182f00085b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:44.331573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773cf347-82dd-4c9a-bad9-8719b51c21c8 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:44.368289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:44.373999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:44.377969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:44.385937 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] PATH_INFO: `/v3/users/f0d56189d2404ec88319445b4eb5e9d8` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:44.393607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d56189d2404ec88319445b4eb5e9d8)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:44.411290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:44.539359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1e4d81-54ac-4b74-b90c-182f00085b04 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:44.551005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb8abf-cdb9-453f-83d3-d541442c8a6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:44.552219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb8abf-cdb9-453f-83d3-d541442c8a6b service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:44.552757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb8abf-cdb9-453f-83d3-d541442c8a6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:44.553231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb8abf-cdb9-453f-83d3-d541442c8a6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:44.578371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb8abf-cdb9-453f-83d3-d541442c8a6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:44.580581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb8abf-cdb9-453f-83d3-d541442c8a6b service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:44.605895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 834/1650] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:44.788678 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d56189d2404ec88319445b4eb5e9d8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:44.883528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d56189d2404ec88319445b4eb5e9d8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:44.884234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d56189d2404ec88319445b4eb5e9d8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:44.913543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d56189d2404ec88319445b4eb5e9d8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:44.915335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d56189d2404ec88319445b4eb5e9d8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:44.940096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 817/1651] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:29:43 2022] DELETE /identity/v3/users/f0d56189d2404ec88319445b4eb5e9d8 => generated 0 bytes in 1043 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:46.060646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb8abf-cdb9-453f-83d3-d541442c8a6b service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:46.062566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-aabb8abf-cdb9-453f-83d3-d541442c8a6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:46.091449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb8abf-cdb9-453f-83d3-d541442c8a6b service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:46.236363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:46.238030 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:46.238750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:46.239215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] PATH_INFO: `/v3/projects/6df4753bdb2e4557afbc2e9434fc12e0` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:46.242281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df4753bdb2e4557afbc2e9434fc12e0)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:46.253954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:46.426009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:46.446946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df4753bdb2e4557afbc2e9434fc12e0', 'request_id': 'req-f670369a-c5a7-443a-9778-d0b5bbddac16'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:46.469994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df4753bdb2e4557afbc2e9434fc12e0', 'request_id': 'req-f670369a-c5a7-443a-9778-d0b5bbddac16'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:46.547438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df4753bdb2e4557afbc2e9434fc12e0', 'request_id': 'req-f670369a-c5a7-443a-9778-d0b5bbddac16'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:46.564380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] The token cache is being invalidate because project 6df4753bdb2e4557afbc2e9434fc12e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:29:46.564819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:46.565968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:46.572635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 835/1652] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:29:46 2022] DELETE /identity/v3/projects/6df4753bdb2e4557afbc2e9434fc12e0 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:29:46.579822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:46.601180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:46.658935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a63d1-e5c5-4f77-b267-db7aa656336e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:46.965278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:47.004171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:47.005258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:47.006188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] PATH_INFO: `/v3/users/47c320152ed047bb8b3ac345de78062c` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:47.009962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c320152ed047bb8b3ac345de78062c)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:47.014708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:47.289753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:47.293623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:47.345949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670369a-c5a7-443a-9778-d0b5bbddac16 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:47.369831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:47.370510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:47.370798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:47.371078 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] PATH_INFO: `/v3/projects/156eb6f806034087a60316940f2286a0` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:47.374215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=156eb6f806034087a60316940f2286a0)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:47.385401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:47.419730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c320152ed047bb8b3ac345de78062c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:47.440231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:47.567220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c320152ed047bb8b3ac345de78062c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:47.569785 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c320152ed047bb8b3ac345de78062c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:47.647815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156eb6f806034087a60316940f2286a0', 'request_id': 'req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:47.654888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c320152ed047bb8b3ac345de78062c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:47.665001 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae50b6e4-a9ef-4490-ba57-9964f60425a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c320152ed047bb8b3ac345de78062c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:47.714916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156eb6f806034087a60316940f2286a0', 'request_id': 'req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:47.784671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 818/1653] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:29:46 2022] DELETE /identity/v3/users/47c320152ed047bb8b3ac345de78062c => generated 0 bytes in 1206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:47.835033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156eb6f806034087a60316940f2286a0', 'request_id': 'req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:47.877002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] The token cache is being invalidate because project 156eb6f806034087a60316940f2286a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:29:47.877668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:47.884712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8fae23f6-64ce-4a93-b6d8-05dcf6b9bd1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:47.894684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 836/1654] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:29:47 2022] DELETE /identity/v3/projects/156eb6f806034087a60316940f2286a0 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:47.903921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0929f4-da6e-46ba-9052-6eba1ced6622 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:47.904477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0929f4-da6e-46ba-9052-6eba1ced6622 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:47.912842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0929f4-da6e-46ba-9052-6eba1ced6622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:48.004754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-3a0929f4-da6e-46ba-9052-6eba1ced6622 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:48.006940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3a0929f4-da6e-46ba-9052-6eba1ced6622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:48.252480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 819/1655] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:48.255295 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60564d59-859e-478e-bb64-3b949c67f9d1 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:48.258403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60564d59-859e-478e-bb64-3b949c67f9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:48.259004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60564d59-859e-478e-bb64-3b949c67f9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:48.367056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-60564d59-859e-478e-bb64-3b949c67f9d1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:48.368438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-60564d59-859e-478e-bb64-3b949c67f9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:48.394073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0929f4-da6e-46ba-9052-6eba1ced6622 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:48.394608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 837/1656] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:29:48.420280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0929f4-da6e-46ba-9052-6eba1ced6622 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:48.435997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc9f57-0c5b-4327-a537-589c06b8026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:48.436771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc9f57-0c5b-4327-a537-589c06b8026a admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:48.437653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc9f57-0c5b-4327-a537-589c06b8026a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:48.437953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc9f57-0c5b-4327-a537-589c06b8026a admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:48.438717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc9f57-0c5b-4327-a537-589c06b8026a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:29:48.439027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc9f57-0c5b-4327-a537-589c06b8026a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:48.444581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc9f57-0c5b-4327-a537-589c06b8026a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:48.476891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 820/1657] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:29:48.481462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60564d59-859e-478e-bb64-3b949c67f9d1 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:48.495384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60564d59-859e-478e-bb64-3b949c67f9d1 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:48.505022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:48.509381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a35c6a-4fc8-433a-85e4-a45164409218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:48.510399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a35c6a-4fc8-433a-85e4-a45164409218 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:48.510888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a35c6a-4fc8-433a-85e4-a45164409218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:48.511403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a35c6a-4fc8-433a-85e4-a45164409218 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:48.512435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a35c6a-4fc8-433a-85e4-a45164409218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:48.516307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a35c6a-4fc8-433a-85e4-a45164409218 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:48.744371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc9f57-0c5b-4327-a537-589c06b8026a admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:48.752354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 838/1658] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:48 2022] POST /identity/v3/projects => generated 385 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:29:48.767235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc9f57-0c5b-4327-a537-589c06b8026a admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:48.969484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b052c65-b51f-4aba-9368-b701c503e11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:48.971987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b052c65-b51f-4aba-9368-b701c503e11f admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:48.971987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b052c65-b51f-4aba-9368-b701c503e11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:48.971987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b052c65-b51f-4aba-9368-b701c503e11f admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:48.971987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b052c65-b51f-4aba-9368-b701c503e11f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:48.978484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b052c65-b51f-4aba-9368-b701c503e11f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:49.120083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 821/1659] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:48 2022] POST /identity/v3/users => generated 293 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:29:49.125845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a35c6a-4fc8-433a-85e4-a45164409218 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:49.146140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a35c6a-4fc8-433a-85e4-a45164409218 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:49.165899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f3bd2-740c-451d-99af-b4d6f86dd539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:49.170268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f3bd2-740c-451d-99af-b4d6f86dd539 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:49.173308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f3bd2-740c-451d-99af-b4d6f86dd539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:49.176720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f3bd2-740c-451d-99af-b4d6f86dd539 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:49.178109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f3bd2-740c-451d-99af-b4d6f86dd539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:49.200900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f3bd2-740c-451d-99af-b4d6f86dd539 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:49.234743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 839/1660] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:49.245650 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b052c65-b51f-4aba-9368-b701c503e11f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:49.253442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b052c65-b51f-4aba-9368-b701c503e11f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:49.272532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c2134-6311-437d-88a4-9f9ce7f448b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:49.273824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c2134-6311-437d-88a4-9f9ce7f448b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:49.274520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c2134-6311-437d-88a4-9f9ce7f448b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:49.275026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c2134-6311-437d-88a4-9f9ce7f448b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:49.276076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c2134-6311-437d-88a4-9f9ce7f448b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:49.278542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c2134-6311-437d-88a4-9f9ce7f448b1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:49.321232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 822/1661] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:49.323730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f3bd2-740c-451d-99af-b4d6f86dd539 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:49.341466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f3bd2-740c-451d-99af-b4d6f86dd539 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:49.367285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e24ff-1a00-42c7-9581-60347230af8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:49.369064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e24ff-1a00-42c7-9581-60347230af8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:49.369573 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e24ff-1a00-42c7-9581-60347230af8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:49.370028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e24ff-1a00-42c7-9581-60347230af8c admin admin] PATH_INFO: `/v3/projects/ece76253a5f44e9894f010c0a9a53a93/users/9d95ffc484094ef593ecb7cdb37713bc/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:49.443627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e24ff-1a00-42c7-9581-60347230af8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece76253a5f44e9894f010c0a9a53a93, user_id=9d95ffc484094ef593ecb7cdb37713bc, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:49.451709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e24ff-1a00-42c7-9581-60347230af8c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:49.589764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 840/1662] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:49 2022] PUT /identity/v3/projects/ece76253a5f44e9894f010c0a9a53a93/users/9d95ffc484094ef593ecb7cdb37713bc/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:49.603334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d4408-8dc0-47fe-bb25-00179362f959 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:49.604031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d4408-8dc0-47fe-bb25-00179362f959 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:49.604482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d4408-8dc0-47fe-bb25-00179362f959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:49.716226 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-7d2d4408-8dc0-47fe-bb25-00179362f959 None None] MFA Rules not processed for user `9d95ffc484094ef593ecb7cdb37713bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:49.720649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7d2d4408-8dc0-47fe-bb25-00179362f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:49.983183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e24ff-1a00-42c7-9581-60347230af8c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:49.984495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 823/1663] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:50.218393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e24ff-1a00-42c7-9581-60347230af8c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:50.233418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2be26e-a3b9-4767-8a7f-1a5dacbd8ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:50.234888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2be26e-a3b9-4767-8a7f-1a5dacbd8ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:50.235382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2be26e-a3b9-4767-8a7f-1a5dacbd8ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:50.235957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2be26e-a3b9-4767-8a7f-1a5dacbd8ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:50.236800 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2be26e-a3b9-4767-8a7f-1a5dacbd8ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:50.239406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2be26e-a3b9-4767-8a7f-1a5dacbd8ae2 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:50.423350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 841/1664] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:49 2022] POST /identity/v3/projects => generated 387 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:50.426539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d4408-8dc0-47fe-bb25-00179362f959 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:50.455740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d4408-8dc0-47fe-bb25-00179362f959 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:50.610485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:50.644847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699c88d-cca4-4e59-884c-076d4f7c5089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:50.646762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699c88d-cca4-4e59-884c-076d4f7c5089 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:50.646762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699c88d-cca4-4e59-884c-076d4f7c5089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:50.646762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699c88d-cca4-4e59-884c-076d4f7c5089 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:50.646762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699c88d-cca4-4e59-884c-076d4f7c5089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:50.649011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699c88d-cca4-4e59-884c-076d4f7c5089 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:50.831200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 824/1665] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:50 2022] POST /identity/v3/users => generated 300 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:50.835655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2be26e-a3b9-4767-8a7f-1a5dacbd8ae2 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:50.856770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2be26e-a3b9-4767-8a7f-1a5dacbd8ae2 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:50.876651 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b280f16-7e7e-4fb1-baaa-17f202ddb188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:50.879999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b280f16-7e7e-4fb1-baaa-17f202ddb188 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:50.881594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b280f16-7e7e-4fb1-baaa-17f202ddb188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:50.882268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b280f16-7e7e-4fb1-baaa-17f202ddb188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:50.886163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b280f16-7e7e-4fb1-baaa-17f202ddb188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:50.891724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b280f16-7e7e-4fb1-baaa-17f202ddb188 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:50.924617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 842/1666] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:50.929315 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699c88d-cca4-4e59-884c-076d4f7c5089 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:50.950337 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699c88d-cca4-4e59-884c-076d4f7c5089 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:50.965060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9697f-c613-4f02-8f8d-4f6547636e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:50.966062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9697f-c613-4f02-8f8d-4f6547636e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:50.966563 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9697f-c613-4f02-8f8d-4f6547636e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:50.968095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9697f-c613-4f02-8f8d-4f6547636e5d admin admin] PATH_INFO: `/v3/projects/2d0be9429f6f4bb78b5a31e2187b85f6/users/19e89eec030941f7aaaa1738f0e15dd9/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:51.021882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a9697f-c613-4f02-8f8d-4f6547636e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0be9429f6f4bb78b5a31e2187b85f6, user_id=19e89eec030941f7aaaa1738f0e15dd9, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:51.021882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a9697f-c613-4f02-8f8d-4f6547636e5d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:51.121010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 825/1667] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:50 2022] PUT /identity/v3/projects/2d0be9429f6f4bb78b5a31e2187b85f6/users/19e89eec030941f7aaaa1738f0e15dd9/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:51.125819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b280f16-7e7e-4fb1-baaa-17f202ddb188 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:51.145433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b280f16-7e7e-4fb1-baaa-17f202ddb188 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:51.178039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:51.279862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4f8e0-dfef-452e-977f-4028022f3af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:51.281052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4f8e0-dfef-452e-977f-4028022f3af9 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:51.281532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4f8e0-dfef-452e-977f-4028022f3af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:51.282026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4f8e0-dfef-452e-977f-4028022f3af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:51.283273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4f8e0-dfef-452e-977f-4028022f3af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:51.286614 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4f8e0-dfef-452e-977f-4028022f3af9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:51.294725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9697f-c613-4f02-8f8d-4f6547636e5d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:51.300611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-37a9697f-c613-4f02-8f8d-4f6547636e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:51.346761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a9697f-c613-4f02-8f8d-4f6547636e5d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:51.357417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 843/1668] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:51.358630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4f8e0-dfef-452e-977f-4028022f3af9 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:51.375440 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:51.378758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:51.379086 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:51.379375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] PATH_INFO: `/v3/projects/f10e431256454066a2e24f6bb917ac89` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:51.386219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10e431256454066a2e24f6bb917ac89)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:51.392155 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:51.412942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4f8e0-dfef-452e-977f-4028022f3af9 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:51.462338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf2a1c-1347-451b-9ce2-3187dc057ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:51.465997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf2a1c-1347-451b-9ce2-3187dc057ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:51.469612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf2a1c-1347-451b-9ce2-3187dc057ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:51.470272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf2a1c-1347-451b-9ce2-3187dc057ba9 admin admin] PATH_INFO: `/v3/projects/2d0be9429f6f4bb78b5a31e2187b85f6/users/19e89eec030941f7aaaa1738f0e15dd9/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:51.475220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf2a1c-1347-451b-9ce2-3187dc057ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0be9429f6f4bb78b5a31e2187b85f6, user_id=19e89eec030941f7aaaa1738f0e15dd9, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:51.485054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf2a1c-1347-451b-9ce2-3187dc057ba9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:51.555750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 844/1669] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:51 2022] PUT /identity/v3/projects/2d0be9429f6f4bb78b5a31e2187b85f6/users/19e89eec030941f7aaaa1738f0e15dd9/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:51.557361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff1f07-e556-409d-8673-8d75009b546a None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:51.557659 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff1f07-e556-409d-8673-8d75009b546a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:51.557948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff1f07-e556-409d-8673-8d75009b546a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:51.639580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a1ff1f07-e556-409d-8673-8d75009b546a None None] MFA Rules not processed for user `1f628503bfd24338b4a351d05a18f126`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:51.642249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a1ff1f07-e556-409d-8673-8d75009b546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:51.743479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10e431256454066a2e24f6bb917ac89', 'request_id': 'req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:51.760370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10e431256454066a2e24f6bb917ac89', 'request_id': 'req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:51.790299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 845/1670] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:51.792306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4087d-59ee-4e6b-b958-170e6f7e1107 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:51.792779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4087d-59ee-4e6b-b958-170e6f7e1107 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:51.793201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4087d-59ee-4e6b-b958-170e6f7e1107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:51.878864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d0e4087d-59ee-4e6b-b958-170e6f7e1107 None None] MFA Rules not processed for user `19e89eec030941f7aaaa1738f0e15dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:51.881233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d0e4087d-59ee-4e6b-b958-170e6f7e1107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:52.056145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10e431256454066a2e24f6bb917ac89', 'request_id': 'req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:52.115465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 846/1671] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:52.120055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c47978-9d6c-4643-90ef-c2f39f1cf13e None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:52.120444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c47978-9d6c-4643-90ef-c2f39f1cf13e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:52.120762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c47978-9d6c-4643-90ef-c2f39f1cf13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:52.152946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a2c47978-9d6c-4643-90ef-c2f39f1cf13e None None] MFA Rules not processed for user `1f628503bfd24338b4a351d05a18f126`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:52.155246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a2c47978-9d6c-4643-90ef-c2f39f1cf13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:52.156451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] The token cache is being invalidate because project f10e431256454066a2e24f6bb917ac89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:29:52.156717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:52.159416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:52.169239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 826/1672] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:29:51 2022] DELETE /identity/v3/projects/f10e431256454066a2e24f6bb917ac89 => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:29:52.170620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:52.184582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:52.292956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 847/1673] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:52.298610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4012c-20fa-4735-a765-056738f2c456 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:52.298610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4012c-20fa-4735-a765-056738f2c456 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:52.299205 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4012c-20fa-4735-a765-056738f2c456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:52.351827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-20b4012c-20fa-4735-a765-056738f2c456 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:52.354680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-20b4012c-20fa-4735-a765-056738f2c456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:52.369143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8020f-1fb2-44d9-84f5-c5b535c87d91 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:52.379845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ca668-8dcf-4e0d-acc3-d67d84db14d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:52.380490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ca668-8dcf-4e0d-acc3-d67d84db14d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:52.380769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ca668-8dcf-4e0d-acc3-d67d84db14d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:52.381117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ca668-8dcf-4e0d-acc3-d67d84db14d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:52.389040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-723ca668-8dcf-4e0d-acc3-d67d84db14d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:52.432038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ca668-8dcf-4e0d-acc3-d67d84db14d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:52.438828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ca668-8dcf-4e0d-acc3-d67d84db14d9 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:52.467147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 827/1674] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:52.468242 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ca668-8dcf-4e0d-acc3-d67d84db14d9 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:52.474419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-723ca668-8dcf-4e0d-acc3-d67d84db14d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:52.532313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 848/1675] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:52.543120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb7d0f-a050-4e07-8989-27fe9f244187 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:52.543746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb7d0f-a050-4e07-8989-27fe9f244187 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:52.544168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb7d0f-a050-4e07-8989-27fe9f244187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:52.590561 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-e3fb7d0f-a050-4e07-8989-27fe9f244187 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:52.592965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e3fb7d0f-a050-4e07-8989-27fe9f244187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:52.626730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:52.630766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 849/1676] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:29:52.631680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb7d0f-a050-4e07-8989-27fe9f244187 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:52.637554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e3fb7d0f-a050-4e07-8989-27fe9f244187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:52.670851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb7d0f-a050-4e07-8989-27fe9f244187 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:52.683043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:52.686698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:52.687343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:52.687886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] PATH_INFO: `/v3/users/9d95ffc484094ef593ecb7cdb37713bc` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:52.688024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ca668-8dcf-4e0d-acc3-d67d84db14d9 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:52.693939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d95ffc484094ef593ecb7cdb37713bc)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:52.699272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:52.704456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb038e6-d70d-411e-b302-19397183cd38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:52.705222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb038e6-d70d-411e-b302-19397183cd38 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:52.705512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb038e6-d70d-411e-b302-19397183cd38 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:52.705795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb038e6-d70d-411e-b302-19397183cd38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:52.713376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0eb038e6-d70d-411e-b302-19397183cd38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:52.739981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb038e6-d70d-411e-b302-19397183cd38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:52.743102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:52.746431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb038e6-d70d-411e-b302-19397183cd38 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:52.769858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 828/1677] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:52.770570 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb038e6-d70d-411e-b302-19397183cd38 service nova] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:52.798312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb038e6-d70d-411e-b302-19397183cd38 service nova] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:52.815727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ff766-aa60-41ae-bc92-6d70f1a025e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:52.816391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ff766-aa60-41ae-bc92-6d70f1a025e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:52.816680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ff766-aa60-41ae-bc92-6d70f1a025e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:52.816961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ff766-aa60-41ae-bc92-6d70f1a025e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:52.817949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ff766-aa60-41ae-bc92-6d70f1a025e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:29:52.818520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ff766-aa60-41ae-bc92-6d70f1a025e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:52.820176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ff766-aa60-41ae-bc92-6d70f1a025e7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:52.863065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 829/1678] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:29:52.864186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ff766-aa60-41ae-bc92-6d70f1a025e7 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:52.890307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ff766-aa60-41ae-bc92-6d70f1a025e7 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:52.897017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187ebb7-e432-42d5-9c50-7edc1aae2c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:52.897661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ebb7-e432-42d5-9c50-7edc1aae2c55 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:52.897940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ebb7-e432-42d5-9c50-7edc1aae2c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:52.898223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187ebb7-e432-42d5-9c50-7edc1aae2c55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:52.898888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187ebb7-e432-42d5-9c50-7edc1aae2c55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:52.901104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187ebb7-e432-42d5-9c50-7edc1aae2c55 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:53.048866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d95ffc484094ef593ecb7cdb37713bc'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:53.053572 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d95ffc484094ef593ecb7cdb37713bc'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:53.058574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d95ffc484094ef593ecb7cdb37713bc'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:53.062748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 830/1679] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:52 2022] POST /identity/v3/projects => generated 365 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:53.063802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187ebb7-e432-42d5-9c50-7edc1aae2c55 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:53.072921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6187ebb7-e432-42d5-9c50-7edc1aae2c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:53.119158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d95ffc484094ef593ecb7cdb37713bc'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:53.154950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8aecad93-0b36-48d1-9311-e931140096cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d95ffc484094ef593ecb7cdb37713bc'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:53.291780 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187ebb7-e432-42d5-9c50-7edc1aae2c55 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:53.301761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 850/1680] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:29:52 2022] DELETE /identity/v3/users/9d95ffc484094ef593ecb7cdb37713bc => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:53.314286 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6d448-d718-4f14-ac05-2bdb6e3bf195 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:53.314750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6d448-d718-4f14-ac05-2bdb6e3bf195 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:53.315184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6d448-d718-4f14-ac05-2bdb6e3bf195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:53.317089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4009d-3fad-4e78-b1c9-1aff5f2dd93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:53.317849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4009d-3fad-4e78-b1c9-1aff5f2dd93f admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:53.318131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4009d-3fad-4e78-b1c9-1aff5f2dd93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:53.318406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4009d-3fad-4e78-b1c9-1aff5f2dd93f admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:53.319111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4009d-3fad-4e78-b1c9-1aff5f2dd93f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:53.322449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4009d-3fad-4e78-b1c9-1aff5f2dd93f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:53.400548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-97d6d448-d718-4f14-ac05-2bdb6e3bf195 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:53.412774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-97d6d448-d718-4f14-ac05-2bdb6e3bf195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:53.450568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 831/1681] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:53 2022] POST /identity/v3/users => generated 283 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:53.452136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4009d-3fad-4e78-b1c9-1aff5f2dd93f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:53.487050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4009d-3fad-4e78-b1c9-1aff5f2dd93f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:53.664087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cdb4be-a038-46bd-9481-8d6501c3fd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:53.664737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdb4be-a038-46bd-9481-8d6501c3fd38 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:53.665023 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdb4be-a038-46bd-9481-8d6501c3fd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:53.665303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cdb4be-a038-46bd-9481-8d6501c3fd38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:53.665987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cdb4be-a038-46bd-9481-8d6501c3fd38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:53.667758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cdb4be-a038-46bd-9481-8d6501c3fd38 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:53.711734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 832/1682] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:53.713285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cdb4be-a038-46bd-9481-8d6501c3fd38 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:53.739783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 851/1683] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:29:53.754743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6d448-d718-4f14-ac05-2bdb6e3bf195 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:53.755834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cdb4be-a038-46bd-9481-8d6501c3fd38 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:53.756600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-97d6d448-d718-4f14-ac05-2bdb6e3bf195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:53.777458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd490b3-50ed-4d10-90ad-25b86391edd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:53.779440 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd490b3-50ed-4d10-90ad-25b86391edd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:53.781950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd490b3-50ed-4d10-90ad-25b86391edd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:53.782792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd490b3-50ed-4d10-90ad-25b86391edd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:53.784445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd490b3-50ed-4d10-90ad-25b86391edd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:53.785710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6d448-d718-4f14-ac05-2bdb6e3bf195 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:53.790940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd490b3-50ed-4d10-90ad-25b86391edd6 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:53.803102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:53.804114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:53.806241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:53.806241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] PATH_INFO: `/v3/users/0cdb50891eb44beba40f2bd9dbeb4759` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:53.807711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cdb50891eb44beba40f2bd9dbeb4759)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:53.811634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:53.832424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 833/1684] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:53.845251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd490b3-50ed-4d10-90ad-25b86391edd6 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:53.857462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4cd490b3-50ed-4d10-90ad-25b86391edd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:54.017859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:54.041691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd490b3-50ed-4d10-90ad-25b86391edd6 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:54.050405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890413f-b4ce-4b56-9d10-87596e372064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:54.051544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890413f-b4ce-4b56-9d10-87596e372064 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:54.051991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890413f-b4ce-4b56-9d10-87596e372064 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:54.052458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3890413f-b4ce-4b56-9d10-87596e372064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:54.085568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890413f-b4ce-4b56-9d10-87596e372064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:54.088560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3890413f-b4ce-4b56-9d10-87596e372064 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:54.116804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cdb50891eb44beba40f2bd9dbeb4759'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:54.118529 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cdb50891eb44beba40f2bd9dbeb4759'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:54.121664 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cdb50891eb44beba40f2bd9dbeb4759'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:54.134817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 834/1685] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:54.136258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890413f-b4ce-4b56-9d10-87596e372064 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:54.145595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cdb50891eb44beba40f2bd9dbeb4759'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:54.162333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3890413f-b4ce-4b56-9d10-87596e372064 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:54.180683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5f4fd-57ad-4326-85ad-97ae273be832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:54.182342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a552a644-1db8-4629-89aa-656e16e0db1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cdb50891eb44beba40f2bd9dbeb4759'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:54.185691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5f4fd-57ad-4326-85ad-97ae273be832 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:54.186814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5f4fd-57ad-4326-85ad-97ae273be832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:54.187356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5f4fd-57ad-4326-85ad-97ae273be832 admin admin] PATH_INFO: `/v3/projects/200bd7bcf81e433d98e3b3fb5bf9ee7c/users/6bd216d1145f464cbdc7bae0745ee2cb/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:54.221133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5f4fd-57ad-4326-85ad-97ae273be832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=200bd7bcf81e433d98e3b3fb5bf9ee7c, user_id=6bd216d1145f464cbdc7bae0745ee2cb, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:54.225180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5f4fd-57ad-4326-85ad-97ae273be832 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:54.297484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 852/1686] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:29:53 2022] DELETE /identity/v3/users/0cdb50891eb44beba40f2bd9dbeb4759 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:54.299347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c7ed1-c52b-4a4d-878f-d5df06772f46 None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:54.299817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c7ed1-c52b-4a4d-878f-d5df06772f46 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:54.300220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c7ed1-c52b-4a4d-878f-d5df06772f46 None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:54.302749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 853/1687] 10.222.0.64 () {54 vars in 912 bytes} [Fri Sep 23 01:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:29:54.305050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba934082-fb8b-4d10-aab2-f5d141e51d1d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:54.305910 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba934082-fb8b-4d10-aab2-f5d141e51d1d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:54.308079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba934082-fb8b-4d10-aab2-f5d141e51d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:54.326892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 835/1688] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:54 2022] PUT /identity/v3/projects/200bd7bcf81e433d98e3b3fb5bf9ee7c/users/6bd216d1145f464cbdc7bae0745ee2cb/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:54.330742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b46787-bb0b-4369-9edc-78538f7b6157 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:54.337548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b46787-bb0b-4369-9edc-78538f7b6157 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:54.344972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b46787-bb0b-4369-9edc-78538f7b6157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:54.383344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:54.426412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ba934082-fb8b-4d10-aab2-f5d141e51d1d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:54.436699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ba934082-fb8b-4d10-aab2-f5d141e51d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:54.520087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b3b46787-bb0b-4369-9edc-78538f7b6157 None None] MFA Rules not processed for user `3b0a7b4f607d4753bf142e1420bcd987`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:54.530335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b3b46787-bb0b-4369-9edc-78538f7b6157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:54.667628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 854/1689] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:54.669938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5d00e-3b9f-4c85-bfcd-e19e655653d1 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:54.676205 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5d00e-3b9f-4c85-bfcd-e19e655653d1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:54.676755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5d00e-3b9f-4c85-bfcd-e19e655653d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:54.679380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 836/1690] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:29:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:54.725156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b46787-bb0b-4369-9edc-78538f7b6157 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:54.727686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b3b46787-bb0b-4369-9edc-78538f7b6157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:54.749766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5dd5d00e-3b9f-4c85-bfcd-e19e655653d1 None None] MFA Rules not processed for user `6bd216d1145f464cbdc7bae0745ee2cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:54.754162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b46787-bb0b-4369-9edc-78538f7b6157 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:54.755392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5dd5d00e-3b9f-4c85-bfcd-e19e655653d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:54.770007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ac62b-52c0-4cda-b87b-67e532972999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:54.770793 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ac62b-52c0-4cda-b87b-67e532972999 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:54.771209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ac62b-52c0-4cda-b87b-67e532972999 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:54.771614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ac62b-52c0-4cda-b87b-67e532972999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:54.793359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ac62b-52c0-4cda-b87b-67e532972999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:54.816037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ac62b-52c0-4cda-b87b-67e532972999 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:54.825374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 837/1691] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:29:54.828401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ac62b-52c0-4cda-b87b-67e532972999 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:54.832573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-919ac62b-52c0-4cda-b87b-67e532972999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:54.860187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ac62b-52c0-4cda-b87b-67e532972999 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:54.874136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e614-d13d-4c9c-b895-c4ee4d74850b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:54.874136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e614-d13d-4c9c-b895-c4ee4d74850b service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:54.875066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e614-d13d-4c9c-b895-c4ee4d74850b service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:54.875136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e614-d13d-4c9c-b895-c4ee4d74850b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:54.892572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4e614-d13d-4c9c-b895-c4ee4d74850b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:54.893696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4e614-d13d-4c9c-b895-c4ee4d74850b service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:54.915065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 838/1692] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:54.927725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 855/1693] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:54.934081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e614-d13d-4c9c-b895-c4ee4d74850b service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:54.938052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c4d4e614-d13d-4c9c-b895-c4ee4d74850b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:54.980637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e614-d13d-4c9c-b895-c4ee4d74850b service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:55.005358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97682568-4c21-4577-8045-2f253f790a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:55.006455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682568-4c21-4577-8045-2f253f790a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:55.007500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682568-4c21-4577-8045-2f253f790a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:55.007500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682568-4c21-4577-8045-2f253f790a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:55.007982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97682568-4c21-4577-8045-2f253f790a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:55.010015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97682568-4c21-4577-8045-2f253f790a9f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:55.196674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 839/1694] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:29:54 2022] POST /identity/v3/projects => generated 365 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:29:55.216090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5d00e-3b9f-4c85-bfcd-e19e655653d1 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:55.242602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5d00e-3b9f-4c85-bfcd-e19e655653d1 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:55.388131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71fdeb-2d20-47f9-a72f-4378db316085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:55.388799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71fdeb-2d20-47f9-a72f-4378db316085 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:55.389139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71fdeb-2d20-47f9-a72f-4378db316085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:55.389443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71fdeb-2d20-47f9-a72f-4378db316085 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:55.390713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71fdeb-2d20-47f9-a72f-4378db316085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:55.403622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71fdeb-2d20-47f9-a72f-4378db316085 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:55.587371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 856/1695] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:29:55 2022] POST /identity/v3/users => generated 289 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:29:55.594312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97682568-4c21-4577-8045-2f253f790a9f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:55.609237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97682568-4c21-4577-8045-2f253f790a9f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:55.624139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245744ab-9fac-4e69-ab98-97aee89565f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:55.625426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245744ab-9fac-4e69-ab98-97aee89565f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:55.625670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245744ab-9fac-4e69-ab98-97aee89565f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:55.626115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245744ab-9fac-4e69-ab98-97aee89565f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:55.627022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245744ab-9fac-4e69-ab98-97aee89565f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:55.633548 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245744ab-9fac-4e69-ab98-97aee89565f3 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:55.667618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 840/1696] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:29:55.674126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71fdeb-2d20-47f9-a72f-4378db316085 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:55.694377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71fdeb-2d20-47f9-a72f-4378db316085 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:55.710863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17454a47-c771-404f-b097-d0202285193b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:55.714619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17454a47-c771-404f-b097-d0202285193b admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:55.715394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17454a47-c771-404f-b097-d0202285193b admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:55.716234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17454a47-c771-404f-b097-d0202285193b admin admin] PATH_INFO: `/v3/projects/33cc0b60680740598c58356c7e638754/users/6295862b47e94c07b197e7f948b9f48f/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:55.764480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17454a47-c771-404f-b097-d0202285193b admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cc0b60680740598c58356c7e638754, user_id=6295862b47e94c07b197e7f948b9f48f, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:55.770649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17454a47-c771-404f-b097-d0202285193b admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:55.940682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 857/1697] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:55 2022] PUT /identity/v3/projects/33cc0b60680740598c58356c7e638754/users/6295862b47e94c07b197e7f948b9f48f/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:29:55.946951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245744ab-9fac-4e69-ab98-97aee89565f3 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:55.967435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245744ab-9fac-4e69-ab98-97aee89565f3 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:56.040569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:56.225644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c08b18-7a24-45b8-88d8-9664b8a7e7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:56.228311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c08b18-7a24-45b8-88d8-9664b8a7e7ea admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:56.228969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c08b18-7a24-45b8-88d8-9664b8a7e7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:56.229914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c08b18-7a24-45b8-88d8-9664b8a7e7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:56.233423 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c08b18-7a24-45b8-88d8-9664b8a7e7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:56.240578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c08b18-7a24-45b8-88d8-9664b8a7e7ea admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:56.271764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17454a47-c771-404f-b097-d0202285193b admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:56.273699 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 841/1698] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:29:56.276907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-17454a47-c771-404f-b097-d0202285193b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:56.287526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c08b18-7a24-45b8-88d8-9664b8a7e7ea admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:56.311420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17454a47-c771-404f-b097-d0202285193b admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:56.324314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c08b18-7a24-45b8-88d8-9664b8a7e7ea admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:56.342009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:56.345646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:56.346854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:56.349802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341af2d-583b-4c39-9cbb-a6f5ca7acbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:56.350798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] PATH_INFO: `/v3/users/a26346680d984c6e8dbb67da0b471ce7` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:56.353236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341af2d-583b-4c39-9cbb-a6f5ca7acbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:56.356011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341af2d-583b-4c39-9cbb-a6f5ca7acbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:56.357070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a26346680d984c6e8dbb67da0b471ce7)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:56.357373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341af2d-583b-4c39-9cbb-a6f5ca7acbf5 admin admin] PATH_INFO: `/v3/projects/33cc0b60680740598c58356c7e638754/users/6295862b47e94c07b197e7f948b9f48f/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:56.366172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341af2d-583b-4c39-9cbb-a6f5ca7acbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cc0b60680740598c58356c7e638754, user_id=6295862b47e94c07b197e7f948b9f48f, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:56.367773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:56.377100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341af2d-583b-4c39-9cbb-a6f5ca7acbf5 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:56.432565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 842/1699] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:29:56 2022] PUT /identity/v3/projects/33cc0b60680740598c58356c7e638754/users/6295862b47e94c07b197e7f948b9f48f/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:56.453737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2be6f3-c334-4bc5-8858-e7022051d173 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:56.454020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2be6f3-c334-4bc5-8858-e7022051d173 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:56.454300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2be6f3-c334-4bc5-8858-e7022051d173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:56.532059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-cd2be6f3-c334-4bc5-8858-e7022051d173 None None] MFA Rules not processed for user `6295862b47e94c07b197e7f948b9f48f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:56.534012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cd2be6f3-c334-4bc5-8858-e7022051d173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:56.771447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:56.804823 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 843/1700] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:29:56.821274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2be6f3-c334-4bc5-8858-e7022051d173 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:56.836325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cd2be6f3-c334-4bc5-8858-e7022051d173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:56.880755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a26346680d984c6e8dbb67da0b471ce7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:56.880755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a26346680d984c6e8dbb67da0b471ce7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:56.885164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a26346680d984c6e8dbb67da0b471ce7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:56.923467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a26346680d984c6e8dbb67da0b471ce7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:56.991838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a26346680d984c6e8dbb67da0b471ce7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:57.088820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2be6f3-c334-4bc5-8858-e7022051d173 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:57.101851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:57.104130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:57.105592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:57.107358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] PATH_INFO: `/v3/projects/ece76253a5f44e9894f010c0a9a53a93` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:57.109783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ece76253a5f44e9894f010c0a9a53a93)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:57.112598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:57.148877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 858/1701] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:29:56 2022] DELETE /identity/v3/users/a26346680d984c6e8dbb67da0b471ce7 => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:57.150404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:57.156254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:57.365309 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04baffb2-effd-4eb8-b94c-1ceb2d4cca9a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:57.386608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ece76253a5f44e9894f010c0a9a53a93', 'request_id': 'req-2e96e21d-a5f8-4d1d-beca-d11428955215'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:57.408099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ece76253a5f44e9894f010c0a9a53a93', 'request_id': 'req-2e96e21d-a5f8-4d1d-beca-d11428955215'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:57.508311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ece76253a5f44e9894f010c0a9a53a93', 'request_id': 'req-2e96e21d-a5f8-4d1d-beca-d11428955215'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:57.533162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] The token cache is being invalidate because project ece76253a5f44e9894f010c0a9a53a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:29:57.533363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503197a-d5bc-4e86-b284-cf9a988f6dd3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:57.534029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:57.535094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:57.535379 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503197a-d5bc-4e86-b284-cf9a988f6dd3 service placement] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:57.535712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503197a-d5bc-4e86-b284-cf9a988f6dd3 service placement] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:57.537064 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 844/1702] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:29:56 2022] DELETE /identity/v3/projects/ece76253a5f44e9894f010c0a9a53a93 => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:57.537840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:57.539459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:57.541851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503197a-d5bc-4e86-b284-cf9a988f6dd3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:57.552083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b503197a-d5bc-4e86-b284-cf9a988f6dd3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:57.596388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503197a-d5bc-4e86-b284-cf9a988f6dd3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:57.599123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b503197a-d5bc-4e86-b284-cf9a988f6dd3 service placement] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:57.613460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:57.614604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 859/1703] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:57.629284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008088a-0bec-42f4-804e-abca4c4a5c7f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:57.629580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008088a-0bec-42f4-804e-abca4c4a5c7f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:57.629987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008088a-0bec-42f4-804e-abca4c4a5c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:57.694376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d008088a-0bec-42f4-804e-abca4c4a5c7f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:57.702649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d008088a-0bec-42f4-804e-abca4c4a5c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:57.806794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96e21d-a5f8-4d1d-beca-d11428955215 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:57.838640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35353c8-ed79-4626-a05e-376bf78c5eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:57.840149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35353c8-ed79-4626-a05e-376bf78c5eac service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:57.840901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35353c8-ed79-4626-a05e-376bf78c5eac service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:57.841384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35353c8-ed79-4626-a05e-376bf78c5eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:57.845186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a35353c8-ed79-4626-a05e-376bf78c5eac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:58.001847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35353c8-ed79-4626-a05e-376bf78c5eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:58.018501 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35353c8-ed79-4626-a05e-376bf78c5eac service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:58.054644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 845/1704] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:58.057017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35353c8-ed79-4626-a05e-376bf78c5eac service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:58.069212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a35353c8-ed79-4626-a05e-376bf78c5eac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:58.104291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 860/1705] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:58.107666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008088a-0bec-42f4-804e-abca4c4a5c7f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:58.114956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d008088a-0bec-42f4-804e-abca4c4a5c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:58.156257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008088a-0bec-42f4-804e-abca4c4a5c7f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:58.170824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:58.171768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:58.173561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35353c8-ed79-4626-a05e-376bf78c5eac service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:58.173753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:58.173990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] PATH_INFO: `/v3/users/19e89eec030941f7aaaa1738f0e15dd9` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:58.176564 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e89eec030941f7aaaa1738f0e15dd9)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:58.182460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:58.212250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:58.216711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:58.218982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:58.220249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] PATH_INFO: `/v3/projects/c44d377f4f9749798e0a1c4ee9a6a5ee` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:58.231250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c44d377f4f9749798e0a1c4ee9a6a5ee)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:58.262732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:58.630641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e89eec030941f7aaaa1738f0e15dd9'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:58.635204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e89eec030941f7aaaa1738f0e15dd9'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:58.635204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e89eec030941f7aaaa1738f0e15dd9'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:58.638756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:58.655342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e89eec030941f7aaaa1738f0e15dd9'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:58.714762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e89eec030941f7aaaa1738f0e15dd9'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:58.817707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c44d377f4f9749798e0a1c4ee9a6a5ee', 'request_id': 'req-6221909c-62c0-4267-ac0f-f228a48a7884'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:58.854491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 861/1706] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:29:58 2022] DELETE /identity/v3/users/19e89eec030941f7aaaa1738f0e15dd9 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:58.856771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:58.865596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:58.872766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c44d377f4f9749798e0a1c4ee9a6a5ee', 'request_id': 'req-6221909c-62c0-4267-ac0f-f228a48a7884'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:59.019519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c44d377f4f9749798e0a1c4ee9a6a5ee', 'request_id': 'req-6221909c-62c0-4267-ac0f-f228a48a7884'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:59.079160 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] The token cache is being invalidate because project c44d377f4f9749798e0a1c4ee9a6a5ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:29:59.079750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:59.109855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:29:59.127748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 846/1707] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:29:58 2022] DELETE /identity/v3/projects/c44d377f4f9749798e0a1c4ee9a6a5ee => generated 0 bytes in 1064 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:29:59.136070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:29:59.142493 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:59.174120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c119e0-baa6-49e6-8533-dfe32cd820c2 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:59.208793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55423e98-b0a4-4fcf-8457-61aeeaf92abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:59.209535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55423e98-b0a4-4fcf-8457-61aeeaf92abd service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:59.210126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55423e98-b0a4-4fcf-8457-61aeeaf92abd service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:59.210442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55423e98-b0a4-4fcf-8457-61aeeaf92abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:59.214755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-55423e98-b0a4-4fcf-8457-61aeeaf92abd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:59.222751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:29:59.436058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55423e98-b0a4-4fcf-8457-61aeeaf92abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:29:59.438154 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55423e98-b0a4-4fcf-8457-61aeeaf92abd service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:29:59.485216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 862/1708] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:29:59.490519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd4e59-7274-4dcb-8b97-9e0b372d7613 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:59.494513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd4e59-7274-4dcb-8b97-9e0b372d7613 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:59.494513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd4e59-7274-4dcb-8b97-9e0b372d7613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:59.556743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-7bcd4e59-7274-4dcb-8b97-9e0b372d7613 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:59.558622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7bcd4e59-7274-4dcb-8b97-9e0b372d7613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:59.567091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221909c-62c0-4267-ac0f-f228a48a7884 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:29:59.575485 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 863/1709] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:59.588200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900169de-b982-405e-ac95-df049000defd None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:59.588484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900169de-b982-405e-ac95-df049000defd None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:59.589156 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900169de-b982-405e-ac95-df049000defd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:59.652803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-900169de-b982-405e-ac95-df049000defd None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:59.658442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-900169de-b982-405e-ac95-df049000defd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:59.747394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 864/1710] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:29:59.749340 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a4da6-c9ba-44ec-9c69-91389ccf1248 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:59.750005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a4da6-c9ba-44ec-9c69-91389ccf1248 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:59.753123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a4da6-c9ba-44ec-9c69-91389ccf1248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:59.831574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d48a4da6-c9ba-44ec-9c69-91389ccf1248 None None] MFA Rules not processed for user `1633ea35eda34ffe884140eaab1ae909`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:29:59.836265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d48a4da6-c9ba-44ec-9c69-91389ccf1248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:59.902586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b0967-2c2c-482d-80ff-38ec7296bf0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:29:59.903761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b0967-2c2c-482d-80ff-38ec7296bf0f service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:59.904768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b0967-2c2c-482d-80ff-38ec7296bf0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:59.904768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b0967-2c2c-482d-80ff-38ec7296bf0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:29:59.911763 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c91b0967-2c2c-482d-80ff-38ec7296bf0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:29:59.945197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 865/1711] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:29:59.947706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c92248-fcb8-42ed-b68d-4cec8041edb8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:29:59.948491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c92248-fcb8-42ed-b68d-4cec8041edb8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:29:59.949284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c92248-fcb8-42ed-b68d-4cec8041edb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:00.038412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d6c92248-fcb8-42ed-b68d-4cec8041edb8 None None] MFA Rules not processed for user `da9978cab3fb4bdda39a2e4718ee2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:00.043003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d6c92248-fcb8-42ed-b68d-4cec8041edb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:00.087393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b0967-2c2c-482d-80ff-38ec7296bf0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:00.090712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b0967-2c2c-482d-80ff-38ec7296bf0f service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:00.123841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 847/1712] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:29:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 995 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:00.129719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b0967-2c2c-482d-80ff-38ec7296bf0f service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:00.188027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b0967-2c2c-482d-80ff-38ec7296bf0f service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:00.204731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb0e35-bc1f-49e1-a2fc-8cd3dc9c12d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:00.205834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb0e35-bc1f-49e1-a2fc-8cd3dc9c12d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:00.207483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb0e35-bc1f-49e1-a2fc-8cd3dc9c12d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:00.208624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb0e35-bc1f-49e1-a2fc-8cd3dc9c12d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:00.211329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb0e35-bc1f-49e1-a2fc-8cd3dc9c12d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:30:00.215986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb0e35-bc1f-49e1-a2fc-8cd3dc9c12d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:00.228264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb0e35-bc1f-49e1-a2fc-8cd3dc9c12d1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:00.267083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:00.298486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 848/1713] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:30:00.304410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ddc64-d522-4a3f-8d14-5415dd54358e None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:00.308170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ddc64-d522-4a3f-8d14-5415dd54358e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:00.308337 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ddc64-d522-4a3f-8d14-5415dd54358e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:00.345681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 866/1714] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:29:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:00.346750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c92248-fcb8-42ed-b68d-4cec8041edb8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:00.354408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d6c92248-fcb8-42ed-b68d-4cec8041edb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:00.431586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-160ddc64-d522-4a3f-8d14-5415dd54358e None None] MFA Rules not processed for user `62f175cb5d1e4ac8a0ee95cf7718837b`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:00.434683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-160ddc64-d522-4a3f-8d14-5415dd54358e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:00.721446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c92248-fcb8-42ed-b68d-4cec8041edb8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:00.745219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c5944e-3dc5-43bf-9ce7-da65ba41e870 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:00.746451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5944e-3dc5-43bf-9ce7-da65ba41e870 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:00.747595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5944e-3dc5-43bf-9ce7-da65ba41e870 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:00.750431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c5944e-3dc5-43bf-9ce7-da65ba41e870 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:00.774162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 849/1715] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:00.775424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ddc64-d522-4a3f-8d14-5415dd54358e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:00.803115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ddc64-d522-4a3f-8d14-5415dd54358e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:00.839772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5944e-3dc5-43bf-9ce7-da65ba41e870 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:00.845537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c31a2-829e-412b-b222-4fb69f9ba772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:00.846366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c31a2-829e-412b-b222-4fb69f9ba772 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:00.846796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c31a2-829e-412b-b222-4fb69f9ba772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:00.846922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c5944e-3dc5-43bf-9ce7-da65ba41e870 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:00.847419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c31a2-829e-412b-b222-4fb69f9ba772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:00.851366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c31a2-829e-412b-b222-4fb69f9ba772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:00.854318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c31a2-829e-412b-b222-4fb69f9ba772 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:00.913817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 867/1716] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:00.916106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4a571-29ec-46a5-80c8-0ab7606be220 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:00.916878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4a571-29ec-46a5-80c8-0ab7606be220 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:00.917665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4a571-29ec-46a5-80c8-0ab7606be220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:00.944939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:00.958756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d4a4a571-29ec-46a5-80c8-0ab7606be220 None None] MFA Rules not processed for user `da9978cab3fb4bdda39a2e4718ee2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:00.960683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d4a4a571-29ec-46a5-80c8-0ab7606be220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:00.980947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 868/1717] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:00.987938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4a571-29ec-46a5-80c8-0ab7606be220 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:00.994304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d4a4a571-29ec-46a5-80c8-0ab7606be220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:01.029501 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 850/1718] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:00 2022] POST /identity/v3/projects => generated 355 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:01.030817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c31a2-829e-412b-b222-4fb69f9ba772 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:01.039854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7a2c31a2-829e-412b-b222-4fb69f9ba772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:01.064999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4a571-29ec-46a5-80c8-0ab7606be220 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:01.258847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c31a2-829e-412b-b222-4fb69f9ba772 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:01.270222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:01.271070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:01.271364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:01.273580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] PATH_INFO: `/v3/projects/2d0be9429f6f4bb78b5a31e2187b85f6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:01.276043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d0be9429f6f4bb78b5a31e2187b85f6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:01.279338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:01.355348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1fde9-e1be-401c-b3c4-27f458095140 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:01.356475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1fde9-e1be-401c-b3c4-27f458095140 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:01.356971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1fde9-e1be-401c-b3c4-27f458095140 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:01.357696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1fde9-e1be-401c-b3c4-27f458095140 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:01.511168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1fde9-e1be-401c-b3c4-27f458095140 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:01.513935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1fde9-e1be-401c-b3c4-27f458095140 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:01.531460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d0be9429f6f4bb78b5a31e2187b85f6', 'request_id': 'req-05122c4f-5c25-4733-9141-5dd3d5e869d3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:01.537994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 869/1719] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:01.540101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1fde9-e1be-401c-b3c4-27f458095140 service nova] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:01.565806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1fde9-e1be-401c-b3c4-27f458095140 service nova] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:01.574003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d0be9429f6f4bb78b5a31e2187b85f6', 'request_id': 'req-05122c4f-5c25-4733-9141-5dd3d5e869d3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:01.637876 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:01.704091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d0be9429f6f4bb78b5a31e2187b85f6', 'request_id': 'req-05122c4f-5c25-4733-9141-5dd3d5e869d3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:01.747297 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] The token cache is being invalidate because project 2d0be9429f6f4bb78b5a31e2187b85f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:30:01.747674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:01.748914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:01.758135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 851/1720] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:30:01 2022] DELETE /identity/v3/projects/2d0be9429f6f4bb78b5a31e2187b85f6 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:01.760479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:01.763370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:01.804640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05a6da-5c66-4f47-b08b-29161aa7a771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:01.806336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05a6da-5c66-4f47-b08b-29161aa7a771 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:01.806939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05a6da-5c66-4f47-b08b-29161aa7a771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:01.807494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05a6da-5c66-4f47-b08b-29161aa7a771 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:01.809160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05a6da-5c66-4f47-b08b-29161aa7a771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:01.812471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05a6da-5c66-4f47-b08b-29161aa7a771 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:01.976870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 870/1721] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:01 2022] POST /identity/v3/users => generated 278 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:01.978544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05a6da-5c66-4f47-b08b-29161aa7a771 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:01.981653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6f05a6da-5c66-4f47-b08b-29161aa7a771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:02.070269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05a6da-5c66-4f47-b08b-29161aa7a771 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:02.097628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05122c4f-5c25-4733-9141-5dd3d5e869d3 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:02.106646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:02.108738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:02.115242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:02.115822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] PATH_INFO: `/v3/projects/eb91993f6c5445abb3780710cc9dba24` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:02.125032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb91993f6c5445abb3780710cc9dba24)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:02.132583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:02.134049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f20f05-1f34-4177-8511-1bb99e9b0a38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:02.137020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f20f05-1f34-4177-8511-1bb99e9b0a38 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:02.137755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f20f05-1f34-4177-8511-1bb99e9b0a38 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:02.138400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f20f05-1f34-4177-8511-1bb99e9b0a38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:02.144959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e1f20f05-1f34-4177-8511-1bb99e9b0a38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:02.432395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f20f05-1f34-4177-8511-1bb99e9b0a38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:02.457396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f20f05-1f34-4177-8511-1bb99e9b0a38 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:02.550483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb91993f6c5445abb3780710cc9dba24', 'request_id': 'req-7023da49-cf63-4ac6-bd41-5c50ec149ad2'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:02.561119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 852/1722] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 802 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:02.568957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cf953-aa3f-4a45-ae07-9cc89e146546 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:02.569238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cf953-aa3f-4a45-ae07-9cc89e146546 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:02.579524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cf953-aa3f-4a45-ae07-9cc89e146546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:02.588059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb91993f6c5445abb3780710cc9dba24', 'request_id': 'req-7023da49-cf63-4ac6-bd41-5c50ec149ad2'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:02.669164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:02.698603 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb91993f6c5445abb3780710cc9dba24', 'request_id': 'req-7023da49-cf63-4ac6-bd41-5c50ec149ad2'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:02.719270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-280cf953-aa3f-4a45-ae07-9cc89e146546 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:02.727472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-280cf953-aa3f-4a45-ae07-9cc89e146546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:02.732335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] The token cache is being invalidate because project eb91993f6c5445abb3780710cc9dba24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:30:02.732633 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:02.745112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:02.762408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 871/1723] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:30:01 2022] DELETE /identity/v3/projects/eb91993f6c5445abb3780710cc9dba24 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:02.762408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:02.771416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:03.078071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 853/1724] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:03.078071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd615a97-5e69-4066-a9fe-bca1d4aa35a7 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:03.078071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd615a97-5e69-4066-a9fe-bca1d4aa35a7 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:03.078071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd615a97-5e69-4066-a9fe-bca1d4aa35a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:03.127829 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023da49-cf63-4ac6-bd41-5c50ec149ad2 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:03.154603 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cb971-1617-4b72-a5d2-cf09bf9ca51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:03.156116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cb971-1617-4b72-a5d2-cf09bf9ca51e admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:03.156700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cb971-1617-4b72-a5d2-cf09bf9ca51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:03.157305 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cb971-1617-4b72-a5d2-cf09bf9ca51e admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:03.158577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-fd615a97-5e69-4066-a9fe-bca1d4aa35a7 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:03.159323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cb971-1617-4b72-a5d2-cf09bf9ca51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:03.162164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cb971-1617-4b72-a5d2-cf09bf9ca51e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:03.167716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fd615a97-5e69-4066-a9fe-bca1d4aa35a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:03.196312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 872/1725] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:03.211835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 854/1726] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:03.213598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd615a97-5e69-4066-a9fe-bca1d4aa35a7 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:03.223605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30466ca4-eae9-4001-b586-6cf883c4ae7b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:03.224054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30466ca4-eae9-4001-b586-6cf883c4ae7b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:03.224547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30466ca4-eae9-4001-b586-6cf883c4ae7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:03.266830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd615a97-5e69-4066-a9fe-bca1d4aa35a7 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:03.284213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae188f8-c50d-4bce-bcc9-65c9b3e1929d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:03.285495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae188f8-c50d-4bce-bcc9-65c9b3e1929d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:03.286096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae188f8-c50d-4bce-bcc9-65c9b3e1929d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:03.286696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae188f8-c50d-4bce-bcc9-65c9b3e1929d admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:03.294176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae188f8-c50d-4bce-bcc9-65c9b3e1929d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:03.309501 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae188f8-c50d-4bce-bcc9-65c9b3e1929d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:03.349419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-30466ca4-eae9-4001-b586-6cf883c4ae7b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:03.351869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-30466ca4-eae9-4001-b586-6cf883c4ae7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:03.366561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 855/1727] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:03.370918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 873/1728] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:03.372366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30466ca4-eae9-4001-b586-6cf883c4ae7b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:03.373258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109d0cf-b1b0-4388-8c8b-003e55634347 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:03.373861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109d0cf-b1b0-4388-8c8b-003e55634347 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:03.374438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109d0cf-b1b0-4388-8c8b-003e55634347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:03.394687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30466ca4-eae9-4001-b586-6cf883c4ae7b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:03.408123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5fa86-f76a-4116-8eb1-2ae0ecf244ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:03.408862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5fa86-f76a-4116-8eb1-2ae0ecf244ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:03.409255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5fa86-f76a-4116-8eb1-2ae0ecf244ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:03.409461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5fa86-f76a-4116-8eb1-2ae0ecf244ae admin admin] PATH_INFO: `/v3/projects/d50e1d80c8844f589c1798fa194f3bb2/users/464fd0776f9942d88c9e3eb060721666/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:03.456639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e109d0cf-b1b0-4388-8c8b-003e55634347 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:03.462859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e109d0cf-b1b0-4388-8c8b-003e55634347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:03.470132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5fa86-f76a-4116-8eb1-2ae0ecf244ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d50e1d80c8844f589c1798fa194f3bb2, user_id=464fd0776f9942d88c9e3eb060721666, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:03.478697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5fa86-f76a-4116-8eb1-2ae0ecf244ae admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:03.532254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 856/1729] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:03.534048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109d0cf-b1b0-4388-8c8b-003e55634347 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:03.564532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109d0cf-b1b0-4388-8c8b-003e55634347 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:03.573127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5121d0b-7783-4d57-9cc0-faab638f1b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:03.574131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5121d0b-7783-4d57-9cc0-faab638f1b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:03.577645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5121d0b-7783-4d57-9cc0-faab638f1b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:03.578396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5121d0b-7783-4d57-9cc0-faab638f1b5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:03.579540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5121d0b-7783-4d57-9cc0-faab638f1b5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:30:03.580357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5121d0b-7783-4d57-9cc0-faab638f1b5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:03.583610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5121d0b-7783-4d57-9cc0-faab638f1b5a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:03.618744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 857/1730] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:30:03.620505 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5121d0b-7783-4d57-9cc0-faab638f1b5a admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:03.628725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 874/1731] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:03 2022] PUT /identity/v3/projects/d50e1d80c8844f589c1798fa194f3bb2/users/464fd0776f9942d88c9e3eb060721666/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:03.629915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5fa86-f76a-4116-8eb1-2ae0ecf244ae admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:03.695319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5121d0b-7783-4d57-9cc0-faab638f1b5a admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:03.934232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e03bc-754e-4087-8b7d-ffea41345c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:03.935007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e03bc-754e-4087-8b7d-ffea41345c80 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:03.935478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e03bc-754e-4087-8b7d-ffea41345c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:03.935905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e03bc-754e-4087-8b7d-ffea41345c80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:03.937250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e03bc-754e-4087-8b7d-ffea41345c80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:30:03.939200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e03bc-754e-4087-8b7d-ffea41345c80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:03.969691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e03bc-754e-4087-8b7d-ffea41345c80 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:04.001583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 858/1732] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:30:04.003469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9265c-7db8-4750-9d0a-ca2ab713daa4 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:04.003912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9265c-7db8-4750-9d0a-ca2ab713daa4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:04.004251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9265c-7db8-4750-9d0a-ca2ab713daa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:04.070982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5fa86-f76a-4116-8eb1-2ae0ecf244ae admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:04.089325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-eaf9265c-7db8-4750-9d0a-ca2ab713daa4 None None] MFA Rules not processed for user `464fd0776f9942d88c9e3eb060721666`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:04.093588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-eaf9265c-7db8-4750-9d0a-ca2ab713daa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:04.122047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81982e-753f-4bf4-8cbf-a7a106ceef6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:04.125204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81982e-753f-4bf4-8cbf-a7a106ceef6f admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:04.126262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81982e-753f-4bf4-8cbf-a7a106ceef6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:04.126772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81982e-753f-4bf4-8cbf-a7a106ceef6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:04.131592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81982e-753f-4bf4-8cbf-a7a106ceef6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:04.139495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81982e-753f-4bf4-8cbf-a7a106ceef6f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:04.254238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:04.305781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 859/1733] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:04.311769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 875/1734] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:03 2022] POST /identity/v3/projects => generated 385 bytes in 679 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:04.312981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9265c-7db8-4750-9d0a-ca2ab713daa4 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:04.315163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81982e-753f-4bf4-8cbf-a7a106ceef6f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:04.358149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9265c-7db8-4750-9d0a-ca2ab713daa4 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:04.503305 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81982e-753f-4bf4-8cbf-a7a106ceef6f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:04.513494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88f5af-abe0-40b1-a6e0-ff2bbf81d231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:04.513916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f5af-abe0-40b1-a6e0-ff2bbf81d231 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:04.514231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f5af-abe0-40b1-a6e0-ff2bbf81d231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:04.514506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f5af-abe0-40b1-a6e0-ff2bbf81d231 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:04.515495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88f5af-abe0-40b1-a6e0-ff2bbf81d231 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:04.518262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88f5af-abe0-40b1-a6e0-ff2bbf81d231 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:04.534328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ef55b-d05a-4d05-b3b0-4292ed8cc978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:04.537641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ef55b-d05a-4d05-b3b0-4292ed8cc978 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:04.538435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ef55b-d05a-4d05-b3b0-4292ed8cc978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:04.540279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ef55b-d05a-4d05-b3b0-4292ed8cc978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:04.541638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ef55b-d05a-4d05-b3b0-4292ed8cc978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:04.548319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ef55b-d05a-4d05-b3b0-4292ed8cc978 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:04.656603 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 876/1735] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:04 2022] POST /identity/v3/projects => generated 357 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:04.658091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88f5af-abe0-40b1-a6e0-ff2bbf81d231 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:04.705343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 860/1736] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:04 2022] POST /identity/v3/projects => generated 381 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:04.705343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a65296-4c13-4529-a74e-79417b07698c None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:04.705343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a65296-4c13-4529-a74e-79417b07698c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:04.705343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a65296-4c13-4529-a74e-79417b07698c None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:04.708321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 861/1737] 10.222.0.64 () {54 vars in 912 bytes} [Fri Sep 23 01:30:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:30:04.711742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a65296-4c13-4529-a74e-79417b07698c None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:04.839820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88f5af-abe0-40b1-a6e0-ff2bbf81d231 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:04.857105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a29cc-e153-45f0-9047-430e8622a24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:04.857802 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a29cc-e153-45f0-9047-430e8622a24a admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:04.860838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a29cc-e153-45f0-9047-430e8622a24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:04.862433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a29cc-e153-45f0-9047-430e8622a24a admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:04.862433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a29cc-e153-45f0-9047-430e8622a24a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:04.866080 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a65296-4c13-4529-a74e-79417b07698c None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:04.868587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a29cc-e153-45f0-9047-430e8622a24a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:04.880586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbc302-178f-40b0-86d1-d25dafd597df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:04.882390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc302-178f-40b0-86d1-d25dafd597df admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:04.882725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc302-178f-40b0-86d1-d25dafd597df admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:04.883011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acbc302-178f-40b0-86d1-d25dafd597df admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:04.897527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbc302-178f-40b0-86d1-d25dafd597df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:04.905918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acbc302-178f-40b0-86d1-d25dafd597df admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:04.918216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:05.054256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 862/1738] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:04 2022] POST /identity/v3/users => generated 285 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:05.056688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbc302-178f-40b0-86d1-d25dafd597df admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:05.060276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 877/1739] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:04 2022] POST /identity/v3/users => generated 293 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:05.063698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83372b6d-1817-4c6f-812c-e9700508b7e4 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:05.064200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83372b6d-1817-4c6f-812c-e9700508b7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:05.064640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83372b6d-1817-4c6f-812c-e9700508b7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:05.106394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acbc302-178f-40b0-86d1-d25dafd597df admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:05.129134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588ca79-7c38-4bd5-9e3e-f84b9da6aa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:05.131544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-83372b6d-1817-4c6f-812c-e9700508b7e4 None None] MFA Rules not processed for user `3b0a7b4f607d4753bf142e1420bcd987`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:05.133355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-83372b6d-1817-4c6f-812c-e9700508b7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:05.135255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588ca79-7c38-4bd5-9e3e-f84b9da6aa47 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:05.138576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588ca79-7c38-4bd5-9e3e-f84b9da6aa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:05.139300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c588ca79-7c38-4bd5-9e3e-f84b9da6aa47 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:05.141021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588ca79-7c38-4bd5-9e3e-f84b9da6aa47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:05.144544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c588ca79-7c38-4bd5-9e3e-f84b9da6aa47 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:05.164135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:05.285051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 863/1740] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:05 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:05.285051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588ca79-7c38-4bd5-9e3e-f84b9da6aa47 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:05.294029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 878/1741] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:30:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:05.295995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83372b6d-1817-4c6f-812c-e9700508b7e4 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:05.322218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c588ca79-7c38-4bd5-9e3e-f84b9da6aa47 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:05.340706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc53392-dfe8-4d12-9a68-f61c24a52c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:05.343424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc53392-dfe8-4d12-9a68-f61c24a52c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:05.345764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc53392-dfe8-4d12-9a68-f61c24a52c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:05.351701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc53392-dfe8-4d12-9a68-f61c24a52c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:05.352042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83372b6d-1817-4c6f-812c-e9700508b7e4 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:05.355519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc53392-dfe8-4d12-9a68-f61c24a52c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:05.359129 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc53392-dfe8-4d12-9a68-f61c24a52c6c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:05.377603 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d4a68-ff1d-4515-98ae-f095131feac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:05.378741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d4a68-ff1d-4515-98ae-f095131feac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:05.379231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d4a68-ff1d-4515-98ae-f095131feac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:05.380298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d4a68-ff1d-4515-98ae-f095131feac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:05.381674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d4a68-ff1d-4515-98ae-f095131feac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:05.385135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d4a68-ff1d-4515-98ae-f095131feac0 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:05.396327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 864/1742] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:05.397711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc53392-dfe8-4d12-9a68-f61c24a52c6c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:05.415206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 879/1743] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:05.417255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d4a68-ff1d-4515-98ae-f095131feac0 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:05.424760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-475d4a68-ff1d-4515-98ae-f095131feac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:05.428859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc53392-dfe8-4d12-9a68-f61c24a52c6c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:05.442363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7cf24-3e7e-481b-a428-b58beb677b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:05.443702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7cf24-3e7e-481b-a428-b58beb677b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:05.444354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7cf24-3e7e-481b-a428-b58beb677b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:05.444978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7cf24-3e7e-481b-a428-b58beb677b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:05.447505 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7cf24-3e7e-481b-a428-b58beb677b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:05.453587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7cf24-3e7e-481b-a428-b58beb677b42 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:05.455023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d4a68-ff1d-4515-98ae-f095131feac0 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:05.463217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e776-b133-4488-bad4-bc4d751dd142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:05.465225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e776-b133-4488-bad4-bc4d751dd142 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:05.466015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e776-b133-4488-bad4-bc4d751dd142 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:05.466988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e776-b133-4488-bad4-bc4d751dd142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:05.479107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 865/1744] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:05.481341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7cf24-3e7e-481b-a428-b58beb677b42 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:05.485691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8e776-b133-4488-bad4-bc4d751dd142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:05.489406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8e776-b133-4488-bad4-bc4d751dd142 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:05.498142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7cf24-3e7e-481b-a428-b58beb677b42 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:05.506383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915769d5-ccc1-4094-b898-c8a6e27025d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:05.507959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915769d5-ccc1-4094-b898-c8a6e27025d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:05.508695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915769d5-ccc1-4094-b898-c8a6e27025d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:05.509454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915769d5-ccc1-4094-b898-c8a6e27025d0 admin admin] PATH_INFO: `/v3/projects/01bf2bb45236430d98a04cb5b30d423d/users/54cdf776a23a435f8a71daa39e4beaad/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:05.509648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 880/1745] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:05.511654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e776-b133-4488-bad4-bc4d751dd142 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:05.535493 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915769d5-ccc1-4094-b898-c8a6e27025d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bf2bb45236430d98a04cb5b30d423d, user_id=54cdf776a23a435f8a71daa39e4beaad, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:05.540100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915769d5-ccc1-4094-b898-c8a6e27025d0 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:05.552819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e776-b133-4488-bad4-bc4d751dd142 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:05.562874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3841d-d7d7-4c6c-8f97-1d26a22f21e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:05.564152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3841d-d7d7-4c6c-8f97-1d26a22f21e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:05.564671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3841d-d7d7-4c6c-8f97-1d26a22f21e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:05.565200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3841d-d7d7-4c6c-8f97-1d26a22f21e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:05.566474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3841d-d7d7-4c6c-8f97-1d26a22f21e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:05.570544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3841d-d7d7-4c6c-8f97-1d26a22f21e8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:05.599763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 881/1746] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:05.601461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3841d-d7d7-4c6c-8f97-1d26a22f21e8 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:05.622726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3841d-d7d7-4c6c-8f97-1d26a22f21e8 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:05.654577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b157f1-5a75-4aba-96db-285f6c27eb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:05.656969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b157f1-5a75-4aba-96db-285f6c27eb09 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:05.662060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b157f1-5a75-4aba-96db-285f6c27eb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:05.663607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 866/1747] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:05 2022] PUT /identity/v3/projects/01bf2bb45236430d98a04cb5b30d423d/users/54cdf776a23a435f8a71daa39e4beaad/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:05.665169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915769d5-ccc1-4094-b898-c8a6e27025d0 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:05.665799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b157f1-5a75-4aba-96db-285f6c27eb09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:05.667194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b157f1-5a75-4aba-96db-285f6c27eb09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:05.680429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b157f1-5a75-4aba-96db-285f6c27eb09 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:05.751307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 882/1748] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:05.753855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b157f1-5a75-4aba-96db-285f6c27eb09 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:05.799355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b157f1-5a75-4aba-96db-285f6c27eb09 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:05.850521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:05.970204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915769d5-ccc1-4094-b898-c8a6e27025d0 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:05.986243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5b77f-3426-471a-b34a-151d4f72cfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:05.988631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5b77f-3426-471a-b34a-151d4f72cfe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:05.989619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5b77f-3426-471a-b34a-151d4f72cfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:05.990509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5b77f-3426-471a-b34a-151d4f72cfe1 admin admin] PATH_INFO: `/v3/projects/573ed517b2c549ba95a552153718ba99/users/5d20bf2f17cf420ba81e7fe0d22dbf15/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:06.117189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5b77f-3426-471a-b34a-151d4f72cfe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=573ed517b2c549ba95a552153718ba99, user_id=5d20bf2f17cf420ba81e7fe0d22dbf15, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:06.130588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c5b77f-3426-471a-b34a-151d4f72cfe1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:06.185931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d281f3b-293e-4d27-9258-0b9f6834e225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:06.190274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d281f3b-293e-4d27-9258-0b9f6834e225 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:06.190748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d281f3b-293e-4d27-9258-0b9f6834e225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:06.195277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d281f3b-293e-4d27-9258-0b9f6834e225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:06.196322 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d281f3b-293e-4d27-9258-0b9f6834e225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:06.199612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d281f3b-293e-4d27-9258-0b9f6834e225 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:06.258755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 883/1749] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:06.258755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d281f3b-293e-4d27-9258-0b9f6834e225 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:06.259380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 867/1750] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:05 2022] PUT /identity/v3/projects/573ed517b2c549ba95a552153718ba99/users/5d20bf2f17cf420ba81e7fe0d22dbf15/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:06.263008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d845e4-25cb-4e5d-ac66-c2b241176921 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:06.264143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d845e4-25cb-4e5d-ac66-c2b241176921 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:06.265752 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d845e4-25cb-4e5d-ac66-c2b241176921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:06.285318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d281f3b-293e-4d27-9258-0b9f6834e225 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:06.459483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-c8d845e4-25cb-4e5d-ac66-c2b241176921 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:06.472119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c8d845e4-25cb-4e5d-ac66-c2b241176921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:06.514512 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:06.628487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9c431-bc2d-4d27-8736-2eb9c7e3c602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:06.629694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c431-bc2d-4d27-8736-2eb9c7e3c602 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:06.630006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c431-bc2d-4d27-8736-2eb9c7e3c602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:06.630268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9c431-bc2d-4d27-8736-2eb9c7e3c602 admin admin] PATH_INFO: `/v3/projects/21d24840bff043b9ab4d9450c4d26281/users/1bb31b86887c41579dcb28e3fd1b2776/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:06.768838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:06.793316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9c431-bc2d-4d27-8736-2eb9c7e3c602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d24840bff043b9ab4d9450c4d26281, user_id=1bb31b86887c41579dcb28e3fd1b2776, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:06.807105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9c431-bc2d-4d27-8736-2eb9c7e3c602 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:06.960370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 868/1751] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:30:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 700 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:06.961857 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d845e4-25cb-4e5d-ac66-c2b241176921 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:07.001805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d845e4-25cb-4e5d-ac66-c2b241176921 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:07.009589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 884/1752] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:06 2022] PUT /identity/v3/projects/21d24840bff043b9ab4d9450c4d26281/users/1bb31b86887c41579dcb28e3fd1b2776/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:07.020168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb262900-c448-410b-8fb5-934d930ae851 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:07.020546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb262900-c448-410b-8fb5-934d930ae851 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:07.023879 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb262900-c448-410b-8fb5-934d930ae851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:07.238509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-bb262900-c448-410b-8fb5-934d930ae851 None None] MFA Rules not processed for user `5d20bf2f17cf420ba81e7fe0d22dbf15`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:07.249734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bb262900-c448-410b-8fb5-934d930ae851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:07.363492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d30fb-28c2-482f-8b60-c2bdd0913538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:07.364280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d30fb-28c2-482f-8b60-c2bdd0913538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:07.364639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d30fb-28c2-482f-8b60-c2bdd0913538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:07.364933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d30fb-28c2-482f-8b60-c2bdd0913538 admin admin] PATH_INFO: `/v3/projects/01bf2bb45236430d98a04cb5b30d423d/users/54cdf776a23a435f8a71daa39e4beaad/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:07.373122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d30fb-28c2-482f-8b60-c2bdd0913538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01bf2bb45236430d98a04cb5b30d423d, user_id=54cdf776a23a435f8a71daa39e4beaad, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:07.376265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d30fb-28c2-482f-8b60-c2bdd0913538 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:07.459033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 869/1753] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:06 2022] PUT /identity/v3/projects/01bf2bb45236430d98a04cb5b30d423d/users/54cdf776a23a435f8a71daa39e4beaad/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:07.461736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045b5c6-3fa0-40e4-839a-0d6ed54a4115 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:07.471839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045b5c6-3fa0-40e4-839a-0d6ed54a4115 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:07.472577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045b5c6-3fa0-40e4-839a-0d6ed54a4115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:07.595108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 885/1754] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 580 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:07.595108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de1ebd-8bf2-4a38-bc01-e983d93bc720 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:07.595108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de1ebd-8bf2-4a38-bc01-e983d93bc720 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:07.595403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de1ebd-8bf2-4a38-bc01-e983d93bc720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:07.618157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e045b5c6-3fa0-40e4-839a-0d6ed54a4115 None None] MFA Rules not processed for user `1bb31b86887c41579dcb28e3fd1b2776`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:07.623436 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e045b5c6-3fa0-40e4-839a-0d6ed54a4115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:07.757204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-27de1ebd-8bf2-4a38-bc01-e983d93bc720 None None] MFA Rules not processed for user `54cdf776a23a435f8a71daa39e4beaad`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:07.759187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-27de1ebd-8bf2-4a38-bc01-e983d93bc720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:07.965596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 886/1755] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:07 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:07.968161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de1ebd-8bf2-4a38-bc01-e983d93bc720 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:08.008064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:08.019395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 870/1756] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 549 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:08.019395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045b5c6-3fa0-40e4-839a-0d6ed54a4115 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:08.019395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e045b5c6-3fa0-40e4-839a-0d6ed54a4115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:08.120883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:08.176650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de1ebd-8bf2-4a38-bc01-e983d93bc720 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:08.193311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e16cd-8dc1-439d-9355-a9d01245a54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:08.193311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e16cd-8dc1-439d-9355-a9d01245a54a admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:08.193604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e16cd-8dc1-439d-9355-a9d01245a54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:08.193880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e16cd-8dc1-439d-9355-a9d01245a54a admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:08.194698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e16cd-8dc1-439d-9355-a9d01245a54a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:08.197275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e16cd-8dc1-439d-9355-a9d01245a54a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:08.218085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045b5c6-3fa0-40e4-839a-0d6ed54a4115 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:08.260895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10bb4f-60aa-4307-8939-5a2fb668d148 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:08.260895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10bb4f-60aa-4307-8939-5a2fb668d148 service swift] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:08.260895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10bb4f-60aa-4307-8939-5a2fb668d148 service swift] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:08.260895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10bb4f-60aa-4307-8939-5a2fb668d148 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:08.396547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 887/1757] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:07 2022] POST /identity/v3/projects => generated 387 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:08.396547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e16cd-8dc1-439d-9355-a9d01245a54a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:08.411189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bb4e16cd-8dc1-439d-9355-a9d01245a54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:08.530097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10bb4f-60aa-4307-8939-5a2fb668d148 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:08.535716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10bb4f-60aa-4307-8939-5a2fb668d148 service swift] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:08.588485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 871/1758] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:30:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:30:08.594730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10bb4f-60aa-4307-8939-5a2fb668d148 service swift] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:08.638219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10bb4f-60aa-4307-8939-5a2fb668d148 service swift] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:08.668781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e16cd-8dc1-439d-9355-a9d01245a54a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:08.686829 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe9947-fd38-43b0-b807-1a8d18b66b3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:08.688571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe9947-fd38-43b0-b807-1a8d18b66b3c service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:08.689359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe9947-fd38-43b0-b807-1a8d18b66b3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:08.690102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe9947-fd38-43b0-b807-1a8d18b66b3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:08.811378 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860eace-a222-4c98-862e-9db17cbcab58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:08.818066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860eace-a222-4c98-862e-9db17cbcab58 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:08.818066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860eace-a222-4c98-862e-9db17cbcab58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:08.818647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860eace-a222-4c98-862e-9db17cbcab58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:08.820285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860eace-a222-4c98-862e-9db17cbcab58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:08.824932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860eace-a222-4c98-862e-9db17cbcab58 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:09.029215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 872/1759] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:08 2022] POST /identity/v3/projects => generated 379 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:09.029215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860eace-a222-4c98-862e-9db17cbcab58 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:09.113493 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe9947-fd38-43b0-b807-1a8d18b66b3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:09.135199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:09.135199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe9947-fd38-43b0-b807-1a8d18b66b3c service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:09.222889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 888/1760] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:08 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 834 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:09.224332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc5cfe-0fc0-460c-bdbb-caab51efadcd None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:09.224849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc5cfe-0fc0-460c-bdbb-caab51efadcd None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:09.225362 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc5cfe-0fc0-460c-bdbb-caab51efadcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:09.366710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f9bc5cfe-0fc0-460c-bdbb-caab51efadcd None None] MFA Rules not processed for user `1633ea35eda34ffe884140eaab1ae909`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:09.368935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f9bc5cfe-0fc0-460c-bdbb-caab51efadcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:09.403353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860eace-a222-4c98-862e-9db17cbcab58 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:09.596711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecdba00-9ba4-460b-8d55-d3f361a6cfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:09.596711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdba00-9ba4-460b-8d55-d3f361a6cfcc admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:09.596711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdba00-9ba4-460b-8d55-d3f361a6cfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:09.596711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecdba00-9ba4-460b-8d55-d3f361a6cfcc admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:09.599154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecdba00-9ba4-460b-8d55-d3f361a6cfcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:09.599154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecdba00-9ba4-460b-8d55-d3f361a6cfcc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:09.623731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 889/1761] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:09.667236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc5cfe-0fc0-460c-bdbb-caab51efadcd None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:09.713301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc5cfe-0fc0-460c-bdbb-caab51efadcd None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:09.757827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd6e7b-37d5-48c6-8d2f-089b877892ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:09.759667 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd6e7b-37d5-48c6-8d2f-089b877892ea admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:09.765884 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd6e7b-37d5-48c6-8d2f-089b877892ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:09.766490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd6e7b-37d5-48c6-8d2f-089b877892ea admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:09.772136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd6e7b-37d5-48c6-8d2f-089b877892ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:09.779609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:09.790334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 873/1762] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:09 2022] POST /identity/v3/users => generated 294 bytes in 762 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:09.806280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2a4ea-ae30-4f4e-9d4b-2cd888088981 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:09.806775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2a4ea-ae30-4f4e-9d4b-2cd888088981 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:09.807254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2a4ea-ae30-4f4e-9d4b-2cd888088981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:09.807390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd6e7b-37d5-48c6-8d2f-089b877892ea admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:09.917715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-fbe2a4ea-ae30-4f4e-9d4b-2cd888088981 None None] MFA Rules not processed for user `da9978cab3fb4bdda39a2e4718ee2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:09.919973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fbe2a4ea-ae30-4f4e-9d4b-2cd888088981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:10.044502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 890/1763] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:09 2022] POST /identity/v3/users => generated 296 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:10.046925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd6e7b-37d5-48c6-8d2f-089b877892ea admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:10.104954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd6e7b-37d5-48c6-8d2f-089b877892ea admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:10.132943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8464e-9c61-4c09-a79b-5f7257df8393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:10.135395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8464e-9c61-4c09-a79b-5f7257df8393 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:10.136234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8464e-9c61-4c09-a79b-5f7257df8393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:10.136971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8464e-9c61-4c09-a79b-5f7257df8393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:10.138500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8464e-9c61-4c09-a79b-5f7257df8393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:10.142740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8464e-9c61-4c09-a79b-5f7257df8393 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:10.194290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 891/1764] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:10.203040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8464e-9c61-4c09-a79b-5f7257df8393 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:10.354237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 874/1765] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 563 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:10.361272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2a4ea-ae30-4f4e-9d4b-2cd888088981 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:10.447144 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2a4ea-ae30-4f4e-9d4b-2cd888088981 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:10.476502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b43cb2-b8b1-401c-8e42-f0394defb9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:10.477151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b43cb2-b8b1-401c-8e42-f0394defb9de admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:10.477445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b43cb2-b8b1-401c-8e42-f0394defb9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:10.477782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b43cb2-b8b1-401c-8e42-f0394defb9de admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:10.485829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b43cb2-b8b1-401c-8e42-f0394defb9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:10.487585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b43cb2-b8b1-401c-8e42-f0394defb9de admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:10.543591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 875/1766] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:10.546319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b43cb2-b8b1-401c-8e42-f0394defb9de admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:10.559798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8464e-9c61-4c09-a79b-5f7257df8393 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:10.582955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747874ed-58ac-4d20-9883-1bf46ee45e77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:10.583751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747874ed-58ac-4d20-9883-1bf46ee45e77 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:10.584043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747874ed-58ac-4d20-9883-1bf46ee45e77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:10.584325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747874ed-58ac-4d20-9883-1bf46ee45e77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:10.591056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b43cb2-b8b1-401c-8e42-f0394defb9de admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:10.637409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6fe02-cc31-455c-9651-0f65bb27bb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:10.643168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6fe02-cc31-455c-9651-0f65bb27bb1c admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:10.646333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6fe02-cc31-455c-9651-0f65bb27bb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:10.648336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6fe02-cc31-455c-9651-0f65bb27bb1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:10.650812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6fe02-cc31-455c-9651-0f65bb27bb1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:10.656674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6fe02-cc31-455c-9651-0f65bb27bb1c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:10.710640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 876/1767] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:10 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:10.715688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6fe02-cc31-455c-9651-0f65bb27bb1c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:10.773463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6fe02-cc31-455c-9651-0f65bb27bb1c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:10.780817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:10.802249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab172476-3161-4a76-ade4-79214d20f19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:10.803694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab172476-3161-4a76-ade4-79214d20f19e admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:10.804189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab172476-3161-4a76-ade4-79214d20f19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:10.804669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab172476-3161-4a76-ade4-79214d20f19e admin admin] PATH_INFO: `/v3/projects/e6d50f15e4d54c74b32bed01ddfe0117/users/ca446f4af69b4e00b5907aa65c67e68d/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:10.822334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747874ed-58ac-4d20-9883-1bf46ee45e77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:10.830845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747874ed-58ac-4d20-9883-1bf46ee45e77 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:10.869131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 892/1768] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:10.873013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747874ed-58ac-4d20-9883-1bf46ee45e77 service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:10.913190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab172476-3161-4a76-ade4-79214d20f19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d50f15e4d54c74b32bed01ddfe0117, user_id=ca446f4af69b4e00b5907aa65c67e68d, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:10.934027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab172476-3161-4a76-ade4-79214d20f19e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:10.948242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747874ed-58ac-4d20-9883-1bf46ee45e77 service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:10.974781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f840c2f-6b64-4a2b-afe1-be5c904afc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:10.975960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f840c2f-6b64-4a2b-afe1-be5c904afc2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:10.976470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f840c2f-6b64-4a2b-afe1-be5c904afc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:10.977676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f840c2f-6b64-4a2b-afe1-be5c904afc2a admin admin] PATH_INFO: `/v3/projects/dd5ae862be384f3e923dcb7041ae0b95/users/170ecfc06f4841c7ac2d754fa100a720/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:11.053155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f840c2f-6b64-4a2b-afe1-be5c904afc2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd5ae862be384f3e923dcb7041ae0b95, user_id=170ecfc06f4841c7ac2d754fa100a720, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:11.057369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f840c2f-6b64-4a2b-afe1-be5c904afc2a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:11.147081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 877/1769] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:10 2022] PUT /identity/v3/projects/e6d50f15e4d54c74b32bed01ddfe0117/users/ca446f4af69b4e00b5907aa65c67e68d/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:11.154410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab172476-3161-4a76-ade4-79214d20f19e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:11.281105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 893/1770] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:10 2022] PUT /identity/v3/projects/dd5ae862be384f3e923dcb7041ae0b95/users/170ecfc06f4841c7ac2d754fa100a720/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:11.291934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18982254-1913-40a6-984a-7dc74f75bd9e None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:11.292255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18982254-1913-40a6-984a-7dc74f75bd9e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:11.292748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18982254-1913-40a6-984a-7dc74f75bd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:11.504113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-18982254-1913-40a6-984a-7dc74f75bd9e None None] MFA Rules not processed for user `170ecfc06f4841c7ac2d754fa100a720`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:11.505499 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-18982254-1913-40a6-984a-7dc74f75bd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:11.631069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab172476-3161-4a76-ade4-79214d20f19e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:11.646139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:12.273841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 894/1771] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 984 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:12.275504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18982254-1913-40a6-984a-7dc74f75bd9e None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:12.308423 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090541c-8c5f-4f23-966c-c257004793c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:12.310772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090541c-8c5f-4f23-966c-c257004793c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:12.312630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090541c-8c5f-4f23-966c-c257004793c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:12.316004 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090541c-8c5f-4f23-966c-c257004793c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:12.317499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090541c-8c5f-4f23-966c-c257004793c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:12.325245 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090541c-8c5f-4f23-966c-c257004793c0 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:12.332353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:12.418015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 878/1772] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 1265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:12.426224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090541c-8c5f-4f23-966c-c257004793c0 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:12.439817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18982254-1913-40a6-984a-7dc74f75bd9e None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:12.493929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090541c-8c5f-4f23-966c-c257004793c0 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:12.518578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f22113-d693-4492-ac8f-ea47ff044301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:12.520117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f22113-d693-4492-ac8f-ea47ff044301 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:12.520902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37340048-9b03-43bd-b545-abccea9c5020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:12.521729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f22113-d693-4492-ac8f-ea47ff044301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:12.527315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f22113-d693-4492-ac8f-ea47ff044301 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:12.528202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37340048-9b03-43bd-b545-abccea9c5020 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:12.528967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37340048-9b03-43bd-b545-abccea9c5020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:12.529561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37340048-9b03-43bd-b545-abccea9c5020 admin admin] PATH_INFO: `/v3/projects/e6d50f15e4d54c74b32bed01ddfe0117/users/ca446f4af69b4e00b5907aa65c67e68d/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:12.529834 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f22113-d693-4492-ac8f-ea47ff044301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:12.534210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f22113-d693-4492-ac8f-ea47ff044301 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:12.540751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37340048-9b03-43bd-b545-abccea9c5020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d50f15e4d54c74b32bed01ddfe0117, user_id=ca446f4af69b4e00b5907aa65c67e68d, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:12.568493 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37340048-9b03-43bd-b545-abccea9c5020 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:12.653179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 879/1773] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:12 2022] PUT /identity/v3/projects/e6d50f15e4d54c74b32bed01ddfe0117/users/ca446f4af69b4e00b5907aa65c67e68d/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:12.657076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8aa74-ccc4-4dbd-89cc-a227a074a2c9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:12.659829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8aa74-ccc4-4dbd-89cc-a227a074a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:12.661751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8aa74-ccc4-4dbd-89cc-a227a074a2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:12.751495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 895/1774] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:12 2022] POST /identity/v3/projects => generated 385 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:12.756296 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f22113-d693-4492-ac8f-ea47ff044301 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:12.786099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-43f8aa74-ccc4-4dbd-89cc-a227a074a2c9 None None] MFA Rules not processed for user `ca446f4af69b4e00b5907aa65c67e68d`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:12.792130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-43f8aa74-ccc4-4dbd-89cc-a227a074a2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:13.125990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f22113-d693-4492-ac8f-ea47ff044301 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:13.160855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99bf4c-e30a-464f-a21e-2d40cbaddf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:13.160855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99bf4c-e30a-464f-a21e-2d40cbaddf64 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:13.160855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99bf4c-e30a-464f-a21e-2d40cbaddf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:13.160855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99bf4c-e30a-464f-a21e-2d40cbaddf64 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:13.162344 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99bf4c-e30a-464f-a21e-2d40cbaddf64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:13.185425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99bf4c-e30a-464f-a21e-2d40cbaddf64 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:13.471487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 896/1775] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:12 2022] POST /identity/v3/users => generated 299 bytes in 716 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:13.476199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99bf4c-e30a-464f-a21e-2d40cbaddf64 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:13.508886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 880/1776] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 855 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:13.565266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99bf4c-e30a-464f-a21e-2d40cbaddf64 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:13.603823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6af0c7-f3fc-48f3-9d19-43a7d4c3a7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:13.606310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6af0c7-f3fc-48f3-9d19-43a7d4c3a7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:13.616102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6af0c7-f3fc-48f3-9d19-43a7d4c3a7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:13.623592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6af0c7-f3fc-48f3-9d19-43a7d4c3a7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:13.625519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6af0c7-f3fc-48f3-9d19-43a7d4c3a7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:13.629459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6af0c7-f3fc-48f3-9d19-43a7d4c3a7a9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:13.724453 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8aa74-ccc4-4dbd-89cc-a227a074a2c9 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:13.737507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 897/1777] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:13.766609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8aa74-ccc4-4dbd-89cc-a227a074a2c9 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:13.806369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d40867-78ef-415f-a727-e9fc5c624155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:13.811043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d40867-78ef-415f-a727-e9fc5c624155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:13.827952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d40867-78ef-415f-a727-e9fc5c624155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:13.827952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d40867-78ef-415f-a727-e9fc5c624155 admin admin] PATH_INFO: `/v3/projects/9e3321b43dd64b4c88d2beefc5137c6a/users/22f8068c20d24468870b08517bbb8cf7/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:13.896473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6af0c7-f3fc-48f3-9d19-43a7d4c3a7a9 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:13.900571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bd6af0c7-f3fc-48f3-9d19-43a7d4c3a7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:13.947339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d40867-78ef-415f-a727-e9fc5c624155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3321b43dd64b4c88d2beefc5137c6a, user_id=22f8068c20d24468870b08517bbb8cf7, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:13.964851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d40867-78ef-415f-a727-e9fc5c624155 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:14.096484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 881/1778] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:13 2022] PUT /identity/v3/projects/9e3321b43dd64b4c88d2beefc5137c6a/users/22f8068c20d24468870b08517bbb8cf7/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:30:14.103278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d40867-78ef-415f-a727-e9fc5c624155 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:14.175812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:14.355095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6af0c7-f3fc-48f3-9d19-43a7d4c3a7a9 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:14.711730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d40867-78ef-415f-a727-e9fc5c624155 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:14.729590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd42ec-130a-40bb-9289-efc58e20db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:14.730886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd42ec-130a-40bb-9289-efc58e20db65 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:14.731853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd42ec-130a-40bb-9289-efc58e20db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:14.733862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd42ec-130a-40bb-9289-efc58e20db65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:14.742640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd42ec-130a-40bb-9289-efc58e20db65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:14.747950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd42ec-130a-40bb-9289-efc58e20db65 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:14.797529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 882/1779] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 695 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:14.805502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd42ec-130a-40bb-9289-efc58e20db65 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:14.805809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee365e-c29c-474f-becb-5ff3ad656766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:14.806837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee365e-c29c-474f-becb-5ff3ad656766 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:14.807338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee365e-c29c-474f-becb-5ff3ad656766 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:14.807877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee365e-c29c-474f-becb-5ff3ad656766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:14.880510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd42ec-130a-40bb-9289-efc58e20db65 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:14.909051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29512aa2-5f25-4cd8-8d34-9721b13f8c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:14.914141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29512aa2-5f25-4cd8-8d34-9721b13f8c9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:14.919477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29512aa2-5f25-4cd8-8d34-9721b13f8c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:14.919477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29512aa2-5f25-4cd8-8d34-9721b13f8c9c admin admin] PATH_INFO: `/v3/projects/9e3321b43dd64b4c88d2beefc5137c6a/users/22f8068c20d24468870b08517bbb8cf7/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:14.968001 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29512aa2-5f25-4cd8-8d34-9721b13f8c9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e3321b43dd64b4c88d2beefc5137c6a, user_id=22f8068c20d24468870b08517bbb8cf7, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:15.012600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29512aa2-5f25-4cd8-8d34-9721b13f8c9c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:15.083827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 883/1780] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:14 2022] PUT /identity/v3/projects/9e3321b43dd64b4c88d2beefc5137c6a/users/22f8068c20d24468870b08517bbb8cf7/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:15.087621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76241e0c-f680-431b-9f9b-f0e4c8ec0186 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:15.088643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76241e0c-f680-431b-9f9b-f0e4c8ec0186 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:15.089492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76241e0c-f680-431b-9f9b-f0e4c8ec0186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:15.171854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-76241e0c-f680-431b-9f9b-f0e4c8ec0186 None None] MFA Rules not processed for user `3b0a7b4f607d4753bf142e1420bcd987`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:15.177372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-76241e0c-f680-431b-9f9b-f0e4c8ec0186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:15.241137 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee365e-c29c-474f-becb-5ff3ad656766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:15.252858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfee365e-c29c-474f-becb-5ff3ad656766 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:15.303641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 898/1781] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 1408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:30:15.305064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f383b04-01ca-460f-94dd-7819e41c9418 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:15.305424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f383b04-01ca-460f-94dd-7819e41c9418 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:15.305754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f383b04-01ca-460f-94dd-7819e41c9418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:15.339404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:15.487193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9f383b04-01ca-460f-94dd-7819e41c9418 None None] MFA Rules not processed for user `22f8068c20d24468870b08517bbb8cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:15.489415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9f383b04-01ca-460f-94dd-7819e41c9418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:15.573939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 884/1782] 10.222.0.81 () {58 vars in 1015 bytes} [Fri Sep 23 01:30:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:15.822318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76241e0c-f680-431b-9f9b-f0e4c8ec0186 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:15.855322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76241e0c-f680-431b-9f9b-f0e4c8ec0186 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:15.886545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef44d68-80da-4d7c-b46e-be90b942c841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:15.887601 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef44d68-80da-4d7c-b46e-be90b942c841 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:15.894371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef44d68-80da-4d7c-b46e-be90b942c841 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:15.895266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef44d68-80da-4d7c-b46e-be90b942c841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:15.944977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef44d68-80da-4d7c-b46e-be90b942c841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:15.960041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef44d68-80da-4d7c-b46e-be90b942c841 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:16.016632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 885/1783] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:16.019464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d021c-25db-446a-b47b-8a16f3e5b4ec None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:16.025471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 899/1784] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 721 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:16.031198 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d021c-25db-446a-b47b-8a16f3e5b4ec None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:16.031198 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d021c-25db-446a-b47b-8a16f3e5b4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:16.066947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f383b04-01ca-460f-94dd-7819e41c9418 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:16.152935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-019d021c-25db-446a-b47b-8a16f3e5b4ec None None] MFA Rules not processed for user `62f175cb5d1e4ac8a0ee95cf7718837b`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:16.155984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-019d021c-25db-446a-b47b-8a16f3e5b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:16.301098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:16.442134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f383b04-01ca-460f-94dd-7819e41c9418 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:16.470615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922b792-056a-498f-b4b9-07d6fc1aa79b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:16.475726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922b792-056a-498f-b4b9-07d6fc1aa79b service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:16.476251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922b792-056a-498f-b4b9-07d6fc1aa79b service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:16.476712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922b792-056a-498f-b4b9-07d6fc1aa79b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:16.551575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 886/1785] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 534 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:16.907401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922b792-056a-498f-b4b9-07d6fc1aa79b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:16.938432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922b792-056a-498f-b4b9-07d6fc1aa79b service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:17.021769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 900/1786] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:16 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 956 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:18.051024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d021c-25db-446a-b47b-8a16f3e5b4ec None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:18.084921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d021c-25db-446a-b47b-8a16f3e5b4ec None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:18.093703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46887891-0422-4874-90b5-8b56b81b4275 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:18.095152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46887891-0422-4874-90b5-8b56b81b4275 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:18.095846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46887891-0422-4874-90b5-8b56b81b4275 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:18.096489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46887891-0422-4874-90b5-8b56b81b4275 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:18.163305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46887891-0422-4874-90b5-8b56b81b4275 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:18.168219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46887891-0422-4874-90b5-8b56b81b4275 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:18.197034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 887/1787] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:22.555405 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91570b57-7a28-4aa5-87f3-487313c29d8d None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:22.556526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91570b57-7a28-4aa5-87f3-487313c29d8d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:22.556986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91570b57-7a28-4aa5-87f3-487313c29d8d None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:22.560711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 901/1788] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:30:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:30:22.848930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8db22-202e-45a8-acec-6e201d1952b5 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:22.849696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8db22-202e-45a8-acec-6e201d1952b5 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:22.849866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8db22-202e-45a8-acec-6e201d1952b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:23.006084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-1bc8db22-202e-45a8-acec-6e201d1952b5 None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:23.018531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1bc8db22-202e-45a8-acec-6e201d1952b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:23.503120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 888/1789] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:30:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 656 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:23.815687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91570b57-7a28-4aa5-87f3-487313c29d8d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:23.817629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-91570b57-7a28-4aa5-87f3-487313c29d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:23.895105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91570b57-7a28-4aa5-87f3-487313c29d8d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:23.942401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ff587-18dd-4cfb-b7a7-43321f468cf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:23.943973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ff587-18dd-4cfb-b7a7-43321f468cf2 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:23.945700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ff587-18dd-4cfb-b7a7-43321f468cf2 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:23.947661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ff587-18dd-4cfb-b7a7-43321f468cf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:24.012595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ff587-18dd-4cfb-b7a7-43321f468cf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:24.016870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ff587-18dd-4cfb-b7a7-43321f468cf2 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:24.064739 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 902/1790] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:24.657528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8db22-202e-45a8-acec-6e201d1952b5 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:24.676746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8db22-202e-45a8-acec-6e201d1952b5 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:24.687702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab79e5b-36d4-422a-a25b-c5d72f0fed01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:24.690918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab79e5b-36d4-422a-a25b-c5d72f0fed01 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:24.692032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab79e5b-36d4-422a-a25b-c5d72f0fed01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:24.694036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab79e5b-36d4-422a-a25b-c5d72f0fed01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:25.013582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:25.124852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab79e5b-36d4-422a-a25b-c5d72f0fed01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:25.136316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab79e5b-36d4-422a-a25b-c5d72f0fed01 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:25.225097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 889/1791] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:30:25.676580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ff587-18dd-4cfb-b7a7-43321f468cf2 service nova] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:25.679839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1b1ff587-18dd-4cfb-b7a7-43321f468cf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:25.951943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ff587-18dd-4cfb-b7a7-43321f468cf2 service nova] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:25.966050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:25.967463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:25.969888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:25.970781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] PATH_INFO: `/v3/users/6bd216d1145f464cbdc7bae0745ee2cb` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:25.989693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bd216d1145f464cbdc7bae0745ee2cb)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:25.996200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:26.272990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0bd8d-8522-4a1e-a4b6-27d6872c89d5 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:26.273966 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0bd8d-8522-4a1e-a4b6-27d6872c89d5 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:26.273966 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b0bd8d-8522-4a1e-a4b6-27d6872c89d5 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:26.277199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 890/1792] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:30:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:30:26.357659 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bf17a-4507-4aed-8d03-82211e6c1ddb None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:26.358139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bf17a-4507-4aed-8d03-82211e6c1ddb None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:26.358680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bf17a-4507-4aed-8d03-82211e6c1ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:26.373171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bd216d1145f464cbdc7bae0745ee2cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:26.378676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bd216d1145f464cbdc7bae0745ee2cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:26.380337 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bd216d1145f464cbdc7bae0745ee2cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:26.438212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bd216d1145f464cbdc7bae0745ee2cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:26.460953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8c8bf17a-4507-4aed-8d03-82211e6c1ddb None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:26.460953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8c8bf17a-4507-4aed-8d03-82211e6c1ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:26.480684 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 891/1793] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:30:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:30:26.487049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bd216d1145f464cbdc7bae0745ee2cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:26.496249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:26.519210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8bf17a-4507-4aed-8d03-82211e6c1ddb None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:26.521761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8c8bf17a-4507-4aed-8d03-82211e6c1ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:26.566125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8bf17a-4507-4aed-8d03-82211e6c1ddb None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:26.594412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d8adb-413d-4493-a0f6-1baea25a5bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:26.598727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d8adb-413d-4493-a0f6-1baea25a5bd2 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:26.599728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d8adb-413d-4493-a0f6-1baea25a5bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:26.600373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d8adb-413d-4493-a0f6-1baea25a5bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:26.632700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d8adb-413d-4493-a0f6-1baea25a5bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:26.636721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d8adb-413d-4493-a0f6-1baea25a5bd2 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:26.673511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 892/1794] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:30:26.686944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e321cf7-1a81-49e1-a78e-63416151a335 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:26.687366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e321cf7-1a81-49e1-a78e-63416151a335 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:26.687895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e321cf7-1a81-49e1-a78e-63416151a335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:26.688667 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 903/1795] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:30:25 2022] DELETE /identity/v3/users/6bd216d1145f464cbdc7bae0745ee2cb => generated 0 bytes in 1005 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:26.737815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-2e321cf7-1a81-49e1-a78e-63416151a335 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:26.739912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2e321cf7-1a81-49e1-a78e-63416151a335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:26.924575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 893/1796] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:30:26.938798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:26.944132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:27.148006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113f151-d7f3-495c-8376-0e06293c0d39 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:27.182602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9c2f9-6510-485e-b3be-3851c221876c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:27.183854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9c2f9-6510-485e-b3be-3851c221876c service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:27.184456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9c2f9-6510-485e-b3be-3851c221876c service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:27.184939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9c2f9-6510-485e-b3be-3851c221876c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:27.206105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9c2f9-6510-485e-b3be-3851c221876c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:27.211345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9c2f9-6510-485e-b3be-3851c221876c service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:27.251488 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 904/1797] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:30:31.514336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e321cf7-1a81-49e1-a78e-63416151a335 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:31.525031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2e321cf7-1a81-49e1-a78e-63416151a335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:31.616651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e321cf7-1a81-49e1-a78e-63416151a335 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:31.697984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:31.700750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:31.715772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:31.716420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] PATH_INFO: `/v3/projects/200bd7bcf81e433d98e3b3fb5bf9ee7c` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:31.734817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200bd7bcf81e433d98e3b3fb5bf9ee7c)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:31.767914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:32.196902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200bd7bcf81e433d98e3b3fb5bf9ee7c', 'request_id': 'req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:32.248206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200bd7bcf81e433d98e3b3fb5bf9ee7c', 'request_id': 'req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:32.505381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200bd7bcf81e433d98e3b3fb5bf9ee7c', 'request_id': 'req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:32.596192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] The token cache is being invalidate because project 200bd7bcf81e433d98e3b3fb5bf9ee7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:30:32.598336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:32.600371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:32.612340 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9c2f9-6510-485e-b3be-3851c221876c service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:32.614537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 894/1798] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:30:31 2022] DELETE /identity/v3/projects/200bd7bcf81e433d98e3b3fb5bf9ee7c => generated 0 bytes in 1092 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:32.654488 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9c2f9-6510-485e-b3be-3851c221876c service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:33.041098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:33.043425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:33.046815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:33.048955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] PATH_INFO: `/v3/users/6295862b47e94c07b197e7f948b9f48f` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:33.057164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6295862b47e94c07b197e7f948b9f48f)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:33.066362 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:33.625518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6295862b47e94c07b197e7f948b9f48f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:33.626868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6295862b47e94c07b197e7f948b9f48f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:33.628680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6295862b47e94c07b197e7f948b9f48f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:33.660127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6295862b47e94c07b197e7f948b9f48f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:33.736503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6295862b47e94c07b197e7f948b9f48f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:34.171489 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 905/1799] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:30:32 2022] DELETE /identity/v3/users/6295862b47e94c07b197e7f948b9f48f => generated 0 bytes in 1560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:37.630344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:37.635090 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:37.879867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:37.896414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efad0cc-ded0-460d-9baa-a1f5cd83fa97 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:38.065159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eee1db-54b3-4042-aa8e-27a3c2dfa567 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:38.067186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eee1db-54b3-4042-aa8e-27a3c2dfa567 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:38.067716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eee1db-54b3-4042-aa8e-27a3c2dfa567 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:38.068276 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eee1db-54b3-4042-aa8e-27a3c2dfa567 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:38.075474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-99eee1db-54b3-4042-aa8e-27a3c2dfa567 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:38.289641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eee1db-54b3-4042-aa8e-27a3c2dfa567 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:38.295501 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eee1db-54b3-4042-aa8e-27a3c2dfa567 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:38.336755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 895/1800] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:38.393507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:38.397626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:38.474133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:38.872853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5246147-754d-467d-b3a2-97d67c7bfc07 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:38.912919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:38.922707 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:38.922707 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:38.922707 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] PATH_INFO: `/v3/projects/33cc0b60680740598c58356c7e638754` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:38.930761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33cc0b60680740598c58356c7e638754)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:38.945760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:39.278560 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33cc0b60680740598c58356c7e638754', 'request_id': 'req-00de128a-df2a-4072-a46f-d0dcf8d959b9'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:39.302043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33cc0b60680740598c58356c7e638754', 'request_id': 'req-00de128a-df2a-4072-a46f-d0dcf8d959b9'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:39.452449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33cc0b60680740598c58356c7e638754', 'request_id': 'req-00de128a-df2a-4072-a46f-d0dcf8d959b9'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:39.499726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] The token cache is being invalidate because project 33cc0b60680740598c58356c7e638754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:30:39.501758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:39.505038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-00de128a-df2a-4072-a46f-d0dcf8d959b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:39.517656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 906/1801] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:30:38 2022] DELETE /identity/v3/projects/33cc0b60680740598c58356c7e638754 => generated 0 bytes in 1125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:39.534156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3af19-500b-4b95-ab2f-cfdb423836c2 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:39.534156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3af19-500b-4b95-ab2f-cfdb423836c2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:39.534156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c3af19-500b-4b95-ab2f-cfdb423836c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:39.583255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-35c3af19-500b-4b95-ab2f-cfdb423836c2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:39.585586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-35c3af19-500b-4b95-ab2f-cfdb423836c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:39.882977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 896/1802] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:30:39.904110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13625c30-ed9d-4339-8ce7-5482bc4a51cb None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:39.905468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13625c30-ed9d-4339-8ce7-5482bc4a51cb None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:39.906900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13625c30-ed9d-4339-8ce7-5482bc4a51cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:39.950470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-13625c30-ed9d-4339-8ce7-5482bc4a51cb None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:39.955309 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-13625c30-ed9d-4339-8ce7-5482bc4a51cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:39.971250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 907/1803] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:39.977945 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3af19-500b-4b95-ab2f-cfdb423836c2 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:39.999145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c3af19-500b-4b95-ab2f-cfdb423836c2 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:40.010757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056f11e-0be2-43da-83cd-79c7a44b6d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:40.011774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056f11e-0be2-43da-83cd-79c7a44b6d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:40.012209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056f11e-0be2-43da-83cd-79c7a44b6d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:40.012597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056f11e-0be2-43da-83cd-79c7a44b6d15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:40.013427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056f11e-0be2-43da-83cd-79c7a44b6d15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:30:40.014006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056f11e-0be2-43da-83cd-79c7a44b6d15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:40.016972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056f11e-0be2-43da-83cd-79c7a44b6d15 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:40.051783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 897/1804] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:30:40.052515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13625c30-ed9d-4339-8ce7-5482bc4a51cb None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:40.074667 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13625c30-ed9d-4339-8ce7-5482bc4a51cb None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:40.087034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1712779-4c83-4787-a8d0-a32bed4a1c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:40.087732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1712779-4c83-4787-a8d0-a32bed4a1c67 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:40.088069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1712779-4c83-4787-a8d0-a32bed4a1c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:40.088365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1712779-4c83-4787-a8d0-a32bed4a1c67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:40.089555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1712779-4c83-4787-a8d0-a32bed4a1c67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:40.092235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1712779-4c83-4787-a8d0-a32bed4a1c67 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:40.305850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 908/1805] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:40 2022] POST /identity/v3/projects => generated 367 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:30:40.307971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056f11e-0be2-43da-83cd-79c7a44b6d15 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:40.341420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056f11e-0be2-43da-83cd-79c7a44b6d15 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:40.649044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b79410-7ac4-4e4e-87a6-d04d6574a422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:40.650248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b79410-7ac4-4e4e-87a6-d04d6574a422 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:40.650758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b79410-7ac4-4e4e-87a6-d04d6574a422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:40.651235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b79410-7ac4-4e4e-87a6-d04d6574a422 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:40.652500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b79410-7ac4-4e4e-87a6-d04d6574a422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:40.656889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b79410-7ac4-4e4e-87a6-d04d6574a422 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:40.948831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 898/1806] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:40 2022] POST /identity/v3/users => generated 284 bytes in 642 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:40.954733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1712779-4c83-4787-a8d0-a32bed4a1c67 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:41.009493 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1712779-4c83-4787-a8d0-a32bed4a1c67 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:41.012653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e955a0-1558-42b1-a582-993a4fc03294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:41.013448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e955a0-1558-42b1-a582-993a4fc03294 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:41.013720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e955a0-1558-42b1-a582-993a4fc03294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:41.014001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e955a0-1558-42b1-a582-993a4fc03294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:41.015320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e955a0-1558-42b1-a582-993a4fc03294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:41.019923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e955a0-1558-42b1-a582-993a4fc03294 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:41.070092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 909/1807] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:41.075263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b79410-7ac4-4e4e-87a6-d04d6574a422 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:41.123697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b79410-7ac4-4e4e-87a6-d04d6574a422 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:41.135291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba275ab-21bb-46b7-9408-ff2871abec64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:41.137917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba275ab-21bb-46b7-9408-ff2871abec64 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:41.140193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba275ab-21bb-46b7-9408-ff2871abec64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:41.141036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba275ab-21bb-46b7-9408-ff2871abec64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:41.142562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba275ab-21bb-46b7-9408-ff2871abec64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:41.147898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba275ab-21bb-46b7-9408-ff2871abec64 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:41.180554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 899/1808] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:41 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:41.185383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e955a0-1558-42b1-a582-993a4fc03294 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:41.232382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e955a0-1558-42b1-a582-993a4fc03294 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:41.242649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6d95b-350d-480b-8b4a-a422f8b61ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:41.253165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6d95b-350d-480b-8b4a-a422f8b61ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:41.254074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6d95b-350d-480b-8b4a-a422f8b61ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:41.260981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6d95b-350d-480b-8b4a-a422f8b61ad5 admin admin] PATH_INFO: `/v3/projects/7d433599ff464cd7913949db00685313/users/4076290ee0f8456b892918970f93b1bc/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:41.334050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6d95b-350d-480b-8b4a-a422f8b61ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d433599ff464cd7913949db00685313, user_id=4076290ee0f8456b892918970f93b1bc, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:41.343911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6d95b-350d-480b-8b4a-a422f8b61ad5 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:41.423827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:41.524611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 910/1809] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:41 2022] PUT /identity/v3/projects/7d433599ff464cd7913949db00685313/users/4076290ee0f8456b892918970f93b1bc/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:41.532407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094da52-83fd-487a-aeb8-2fd9bdaed660 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:41.532407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094da52-83fd-487a-aeb8-2fd9bdaed660 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:41.532407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094da52-83fd-487a-aeb8-2fd9bdaed660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:41.653336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-0094da52-83fd-487a-aeb8-2fd9bdaed660 None None] MFA Rules not processed for user `4076290ee0f8456b892918970f93b1bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:41.657679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0094da52-83fd-487a-aeb8-2fd9bdaed660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:41.756084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6d95b-350d-480b-8b4a-a422f8b61ad5 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:41.758595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-39c6d95b-350d-480b-8b4a-a422f8b61ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:41.945133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 900/1810] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:41.945133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094da52-83fd-487a-aeb8-2fd9bdaed660 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:41.988711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6d95b-350d-480b-8b4a-a422f8b61ad5 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:42.008047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094da52-83fd-487a-aeb8-2fd9bdaed660 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:42.010437 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:42.011689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:42.015821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:42.018048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] PATH_INFO: `/v3/users/1bb31b86887c41579dcb28e3fd1b2776` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:42.022785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bb31b86887c41579dcb28e3fd1b2776)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:42.025966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:42.030999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5824-1317-40ab-8905-6d0db14ec370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:42.032715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5824-1317-40ab-8905-6d0db14ec370 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:42.032715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5824-1317-40ab-8905-6d0db14ec370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:42.033107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5824-1317-40ab-8905-6d0db14ec370 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:42.034331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f5824-1317-40ab-8905-6d0db14ec370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:42.049997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f5824-1317-40ab-8905-6d0db14ec370 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:42.302976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 901/1811] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:41 2022] POST /identity/v3/projects => generated 369 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:42.313960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5824-1317-40ab-8905-6d0db14ec370 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:42.514675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bb31b86887c41579dcb28e3fd1b2776'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:42.525456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bb31b86887c41579dcb28e3fd1b2776'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:42.525456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bb31b86887c41579dcb28e3fd1b2776'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:42.559164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bb31b86887c41579dcb28e3fd1b2776'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:42.613561 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0bdf9416-b378-4674-ad22-a35ba1adf4c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bb31b86887c41579dcb28e3fd1b2776'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:42.732685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5824-1317-40ab-8905-6d0db14ec370 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:42.754453 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ac62e-1857-42de-9fd2-633abde53fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:42.757845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ac62e-1857-42de-9fd2-633abde53fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:42.758536 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ac62e-1857-42de-9fd2-633abde53fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:42.759122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ac62e-1857-42de-9fd2-633abde53fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:42.760731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ac62e-1857-42de-9fd2-633abde53fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:42.766353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ac62e-1857-42de-9fd2-633abde53fc4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:42.789172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 911/1812] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:30:41 2022] DELETE /identity/v3/users/1bb31b86887c41579dcb28e3fd1b2776 => generated 0 bytes in 1029 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:42.791849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dae8d-bc53-4d0d-839a-a2eab6083e7b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:42.792389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dae8d-bc53-4d0d-839a-a2eab6083e7b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:42.792910 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731dae8d-bc53-4d0d-839a-a2eab6083e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:42.818604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:42.920113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-731dae8d-bc53-4d0d-839a-a2eab6083e7b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:42.923061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-731dae8d-bc53-4d0d-839a-a2eab6083e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:43.040815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 912/1813] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:43.042543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731dae8d-bc53-4d0d-839a-a2eab6083e7b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:43.050034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-731dae8d-bc53-4d0d-839a-a2eab6083e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:43.093576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 902/1814] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:42 2022] POST /identity/v3/users => generated 291 bytes in 778 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:43.095465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ac62e-1857-42de-9fd2-633abde53fc4 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:43.107090 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f35ac62e-1857-42de-9fd2-633abde53fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:43.164153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ac62e-1857-42de-9fd2-633abde53fc4 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:43.191002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47003cc-e3c9-4b23-b40c-bdde4cee0e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:43.192711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47003cc-e3c9-4b23-b40c-bdde4cee0e88 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:43.193451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47003cc-e3c9-4b23-b40c-bdde4cee0e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:43.194223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47003cc-e3c9-4b23-b40c-bdde4cee0e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:43.199098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47003cc-e3c9-4b23-b40c-bdde4cee0e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:43.208191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47003cc-e3c9-4b23-b40c-bdde4cee0e88 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:43.254333 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 903/1815] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:43.256903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47003cc-e3c9-4b23-b40c-bdde4cee0e88 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:43.294176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731dae8d-bc53-4d0d-839a-a2eab6083e7b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:43.313348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8416-e8fb-4bc8-983f-87a0f3355e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:43.317478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8416-e8fb-4bc8-983f-87a0f3355e74 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:43.318179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8416-e8fb-4bc8-983f-87a0f3355e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:43.320455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f8416-e8fb-4bc8-983f-87a0f3355e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:43.347329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47003cc-e3c9-4b23-b40c-bdde4cee0e88 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:43.367236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45cd97-5c52-4ad3-9faf-000d0b235ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:43.375841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45cd97-5c52-4ad3-9faf-000d0b235ca9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:43.376267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45cd97-5c52-4ad3-9faf-000d0b235ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:43.376596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45cd97-5c52-4ad3-9faf-000d0b235ca9 admin admin] PATH_INFO: `/v3/projects/314224f2e7364d1f93c7143b45a7e065/users/e559326dd25b4152b30be95ac2e0b944/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:43.377718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f8416-e8fb-4bc8-983f-87a0f3355e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:43.388431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f8416-e8fb-4bc8-983f-87a0f3355e74 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:43.432837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 913/1816] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:43.446087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45cd97-5c52-4ad3-9faf-000d0b235ca9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=314224f2e7364d1f93c7143b45a7e065, user_id=e559326dd25b4152b30be95ac2e0b944, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:43.454311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45cd97-5c52-4ad3-9faf-000d0b235ca9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:43.521748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:43.599865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 904/1817] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:43 2022] PUT /identity/v3/projects/314224f2e7364d1f93c7143b45a7e065/users/e559326dd25b4152b30be95ac2e0b944/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:43.604219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8416-e8fb-4bc8-983f-87a0f3355e74 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:43.639036 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f8416-e8fb-4bc8-983f-87a0f3355e74 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:43.849949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82dd5b6-942d-47dc-a24c-63664930ba69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:43.852199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dd5b6-942d-47dc-a24c-63664930ba69 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:43.852695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dd5b6-942d-47dc-a24c-63664930ba69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:43.853271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dd5b6-942d-47dc-a24c-63664930ba69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:43.854192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82dd5b6-942d-47dc-a24c-63664930ba69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:43.857942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82dd5b6-942d-47dc-a24c-63664930ba69 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:43.880507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 914/1818] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:43.880727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45cd97-5c52-4ad3-9faf-000d0b235ca9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:43.902486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45cd97-5c52-4ad3-9faf-000d0b235ca9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:43.919418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306d7b3-71c8-4312-a029-8b6077c0e858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:43.928747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306d7b3-71c8-4312-a029-8b6077c0e858 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:43.928747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306d7b3-71c8-4312-a029-8b6077c0e858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:43.928747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306d7b3-71c8-4312-a029-8b6077c0e858 admin admin] PATH_INFO: `/v3/projects/314224f2e7364d1f93c7143b45a7e065/users/e559326dd25b4152b30be95ac2e0b944/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:43.928747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306d7b3-71c8-4312-a029-8b6077c0e858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=314224f2e7364d1f93c7143b45a7e065, user_id=e559326dd25b4152b30be95ac2e0b944, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:43.928747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306d7b3-71c8-4312-a029-8b6077c0e858 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:43.964385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 905/1819] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:43 2022] PUT /identity/v3/projects/314224f2e7364d1f93c7143b45a7e065/users/e559326dd25b4152b30be95ac2e0b944/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:30:43.989072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b74f-30c4-4abf-8857-0127cd7a6d8d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:43.989840 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b74f-30c4-4abf-8857-0127cd7a6d8d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:43.990533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b74f-30c4-4abf-8857-0127cd7a6d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:44.070284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b751b74f-30c4-4abf-8857-0127cd7a6d8d None None] MFA Rules not processed for user `e559326dd25b4152b30be95ac2e0b944`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:44.074660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b751b74f-30c4-4abf-8857-0127cd7a6d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:44.248265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 915/1820] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:44.405754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306d7b3-71c8-4312-a029-8b6077c0e858 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:44.415855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3306d7b3-71c8-4312-a029-8b6077c0e858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:44.635655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306d7b3-71c8-4312-a029-8b6077c0e858 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:44.646534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfbdaf-045e-4592-bab3-568e15bfe1d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:44.649116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfbdaf-045e-4592-bab3-568e15bfe1d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:44.649902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfbdaf-045e-4592-bab3-568e15bfe1d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:44.651592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfbdaf-045e-4592-bab3-568e15bfe1d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:44.680081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfbdaf-045e-4592-bab3-568e15bfe1d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:44.686121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfbdaf-045e-4592-bab3-568e15bfe1d8 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:44.728532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 906/1821] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:44.897808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b74f-30c4-4abf-8857-0127cd7a6d8d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:44.900422 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b751b74f-30c4-4abf-8857-0127cd7a6d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:44.935349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb7100-c02c-47e3-9ea6-91b1b17f2b99 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:44.936457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb7100-c02c-47e3-9ea6-91b1b17f2b99 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:44.938775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb7100-c02c-47e3-9ea6-91b1b17f2b99 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:44.940052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b74f-30c4-4abf-8857-0127cd7a6d8d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:44.942165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 907/1822] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:30:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 23 01:30:44.949553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e4fa0-db51-4a22-bcc0-79b1ab10df79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:44.950467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e4fa0-db51-4a22-bcc0-79b1ab10df79 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:44.950930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e4fa0-db51-4a22-bcc0-79b1ab10df79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:44.951384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e4fa0-db51-4a22-bcc0-79b1ab10df79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:44.954891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-972e4fa0-db51-4a22-bcc0-79b1ab10df79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:44.968109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12862b-d88d-40c2-8fb7-65e60bc04ebf None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:44.969421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12862b-d88d-40c2-8fb7-65e60bc04ebf None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:44.970100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12862b-d88d-40c2-8fb7-65e60bc04ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:45.008208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-5b12862b-d88d-40c2-8fb7-65e60bc04ebf None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:45.011098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5b12862b-d88d-40c2-8fb7-65e60bc04ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:45.214869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e4fa0-db51-4a22-bcc0-79b1ab10df79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:45.218419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 908/1823] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:30:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:30:45.221467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e4fa0-db51-4a22-bcc0-79b1ab10df79 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:45.253367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12862b-d88d-40c2-8fb7-65e60bc04ebf None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:45.258931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5b12862b-d88d-40c2-8fb7-65e60bc04ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:45.283710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 916/1824] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:45.305596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12862b-d88d-40c2-8fb7-65e60bc04ebf None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:45.322927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23ed2b-c01e-46db-b3cc-bdd936d7ebee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:45.324246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23ed2b-c01e-46db-b3cc-bdd936d7ebee service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:45.324889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23ed2b-c01e-46db-b3cc-bdd936d7ebee service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:45.327515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23ed2b-c01e-46db-b3cc-bdd936d7ebee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:45.354589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23ed2b-c01e-46db-b3cc-bdd936d7ebee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:45.357920 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f23ed2b-c01e-46db-b3cc-bdd936d7ebee service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:45.386348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 909/1825] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:46.444347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e4fa0-db51-4a22-bcc0-79b1ab10df79 service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:46.446968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-972e4fa0-db51-4a22-bcc0-79b1ab10df79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:46.765048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e4fa0-db51-4a22-bcc0-79b1ab10df79 service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:46.800314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:46.807288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:46.809039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:46.815791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] PATH_INFO: `/v3/projects/21d24840bff043b9ab4d9450c4d26281` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:46.821617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d24840bff043b9ab4d9450c4d26281)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:46.829131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:47.227081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d24840bff043b9ab4d9450c4d26281', 'request_id': 'req-896f10e8-3c22-4519-a56b-4f0d26a8459b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:47.266588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d24840bff043b9ab4d9450c4d26281', 'request_id': 'req-896f10e8-3c22-4519-a56b-4f0d26a8459b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:47.507664 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d24840bff043b9ab4d9450c4d26281', 'request_id': 'req-896f10e8-3c22-4519-a56b-4f0d26a8459b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:47.560403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] The token cache is being invalidate because project 21d24840bff043b9ab4d9450c4d26281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:30:47.560897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:47.573454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:47.582875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 917/1826] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:30:46 2022] DELETE /identity/v3/projects/21d24840bff043b9ab4d9450c4d26281 => generated 0 bytes in 1140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:47.589891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23ed2b-c01e-46db-b3cc-bdd936d7ebee service nova] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:47.654911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23ed2b-c01e-46db-b3cc-bdd936d7ebee service nova] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:47.710991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:48.068680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:48.095647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:48.098148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:48.099637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:48.101321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] PATH_INFO: `/v3/users/ca446f4af69b4e00b5907aa65c67e68d` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:48.104560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca446f4af69b4e00b5907aa65c67e68d)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:48.112157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:48.601957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca446f4af69b4e00b5907aa65c67e68d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:48.767100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca446f4af69b4e00b5907aa65c67e68d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:48.767100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca446f4af69b4e00b5907aa65c67e68d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:48.814445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca446f4af69b4e00b5907aa65c67e68d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:48.816505 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca446f4af69b4e00b5907aa65c67e68d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:48.897469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 910/1827] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:30:47 2022] DELETE /identity/v3/users/ca446f4af69b4e00b5907aa65c67e68d => generated 0 bytes in 1305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:30:50.289812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:50.292484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:50.341389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f10e8-3c22-4519-a56b-4f0d26a8459b admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:50.439923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:50.447106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:50.738178 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:50.740099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:50.742646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:50.743276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] PATH_INFO: `/v3/users/5d20bf2f17cf420ba81e7fe0d22dbf15` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:50.762815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d20bf2f17cf420ba81e7fe0d22dbf15)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:50.773315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:50.858371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f467f-26c9-4e40-b44b-ee1e74530471 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:50.885283 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:50.886923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:50.887860 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:50.888722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] PATH_INFO: `/v3/projects/e6d50f15e4d54c74b32bed01ddfe0117` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:50.891513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d50f15e4d54c74b32bed01ddfe0117)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:50.896360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:51.046455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:51.304345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d50f15e4d54c74b32bed01ddfe0117', 'request_id': 'req-fe094228-58cd-4a36-b429-43108b5a5aaa'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:51.337956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d50f15e4d54c74b32bed01ddfe0117', 'request_id': 'req-fe094228-58cd-4a36-b429-43108b5a5aaa'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:51.362199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d20bf2f17cf420ba81e7fe0d22dbf15'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:51.367396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d20bf2f17cf420ba81e7fe0d22dbf15'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:51.369062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d20bf2f17cf420ba81e7fe0d22dbf15'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:51.426016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d20bf2f17cf420ba81e7fe0d22dbf15'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:51.434388 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d50f15e4d54c74b32bed01ddfe0117', 'request_id': 'req-fe094228-58cd-4a36-b429-43108b5a5aaa'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:51.491598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] The token cache is being invalidate because project e6d50f15e4d54c74b32bed01ddfe0117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:30:51.492646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:51.507114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fe094228-58cd-4a36-b429-43108b5a5aaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:51.510307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 911/1828] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:30:50 2022] DELETE /identity/v3/projects/e6d50f15e4d54c74b32bed01ddfe0117 => generated 0 bytes in 1075 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:30:51.520061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fac56-fb45-4abc-ac30-475c78acef28 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:51.520233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fac56-fb45-4abc-ac30-475c78acef28 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:51.520717 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fac56-fb45-4abc-ac30-475c78acef28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:51.525198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-59da8c14-cb13-44a9-b13b-ba3e86d9a2b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d20bf2f17cf420ba81e7fe0d22dbf15'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:51.557419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:51.628690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-716fac56-fb45-4abc-ac30-475c78acef28 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:51.633755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-716fac56-fb45-4abc-ac30-475c78acef28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:51.640115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 918/1829] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:30:50 2022] DELETE /identity/v3/users/5d20bf2f17cf420ba81e7fe0d22dbf15 => generated 0 bytes in 1352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:51.654252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e74e-55a8-43e5-be70-fb633f3f8e0d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:51.658773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e74e-55a8-43e5-be70-fb633f3f8e0d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:51.662946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e74e-55a8-43e5-be70-fb633f3f8e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:51.757513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ba22e74e-55a8-43e5-be70-fb633f3f8e0d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:51.765314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ba22e74e-55a8-43e5-be70-fb633f3f8e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:51.980413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:51.981708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 912/1830] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:30:51.986918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e1c83-2c50-4ff0-bdf8-7562ee827797 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:51.987581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e1c83-2c50-4ff0-bdf8-7562ee827797 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:51.988054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e1c83-2c50-4ff0-bdf8-7562ee827797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:52.015255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 919/1831] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:52.037134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-915e1c83-2c50-4ff0-bdf8-7562ee827797 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:52.040225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-915e1c83-2c50-4ff0-bdf8-7562ee827797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:52.066182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 913/1832] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:30:52.068306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22e74e-55a8-43e5-be70-fb633f3f8e0d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:52.095261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e1c83-2c50-4ff0-bdf8-7562ee827797 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:52.098589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-915e1c83-2c50-4ff0-bdf8-7562ee827797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:52.107328 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22e74e-55a8-43e5-be70-fb633f3f8e0d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:52.133073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9da666-156c-4a9b-9ef9-8a8605ea56f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:52.134186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9da666-156c-4a9b-9ef9-8a8605ea56f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:52.134759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9da666-156c-4a9b-9ef9-8a8605ea56f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:52.135250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9da666-156c-4a9b-9ef9-8a8605ea56f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:52.136641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9da666-156c-4a9b-9ef9-8a8605ea56f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:30:52.137297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9da666-156c-4a9b-9ef9-8a8605ea56f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:52.148454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9da666-156c-4a9b-9ef9-8a8605ea56f5 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:52.199734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 920/1833] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:30:52.207418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9da666-156c-4a9b-9ef9-8a8605ea56f5 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:52.236170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9da666-156c-4a9b-9ef9-8a8605ea56f5 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:52.257231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508c11b-fb9c-4678-b95a-a59c25c23d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:52.258368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508c11b-fb9c-4678-b95a-a59c25c23d58 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:52.259037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508c11b-fb9c-4678-b95a-a59c25c23d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:52.259539 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508c11b-fb9c-4678-b95a-a59c25c23d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:52.261084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508c11b-fb9c-4678-b95a-a59c25c23d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:52.267406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5508c11b-fb9c-4678-b95a-a59c25c23d58 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:52.381839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e1c83-2c50-4ff0-bdf8-7562ee827797 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:52.392820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d59e85-d615-41b8-996d-b6489e7fc78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:52.394000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d59e85-d615-41b8-996d-b6489e7fc78d service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:52.395668 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d59e85-d615-41b8-996d-b6489e7fc78d service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:52.395668 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d59e85-d615-41b8-996d-b6489e7fc78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:52.438076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 921/1834] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:52 2022] POST /identity/v3/projects => generated 363 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:52.445950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508c11b-fb9c-4678-b95a-a59c25c23d58 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:52.453541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d59e85-d615-41b8-996d-b6489e7fc78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:52.459156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d59e85-d615-41b8-996d-b6489e7fc78d service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:52.547119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 914/1835] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:30:52.726948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5508c11b-fb9c-4678-b95a-a59c25c23d58 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:52.746882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657b991-f6d2-4737-8adf-c63c42d17453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:52.750699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657b991-f6d2-4737-8adf-c63c42d17453 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:52.751122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657b991-f6d2-4737-8adf-c63c42d17453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:52.754583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657b991-f6d2-4737-8adf-c63c42d17453 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:52.757439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657b991-f6d2-4737-8adf-c63c42d17453 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:52.767069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657b991-f6d2-4737-8adf-c63c42d17453 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:53.212970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 922/1836] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:52 2022] POST /identity/v3/users => generated 282 bytes in 769 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:53.228188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d59e85-d615-41b8-996d-b6489e7fc78d service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:53.259939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d59e85-d615-41b8-996d-b6489e7fc78d service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:53.281279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646067cc-a91e-4ff8-8ef7-ad28d5a8b2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:53.282389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646067cc-a91e-4ff8-8ef7-ad28d5a8b2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:53.283177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646067cc-a91e-4ff8-8ef7-ad28d5a8b2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:53.283472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646067cc-a91e-4ff8-8ef7-ad28d5a8b2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:53.295824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646067cc-a91e-4ff8-8ef7-ad28d5a8b2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:53.301512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646067cc-a91e-4ff8-8ef7-ad28d5a8b2f2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:53.342487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657b991-f6d2-4737-8adf-c63c42d17453 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:53.353093 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 915/1837] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:53.383045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657b991-f6d2-4737-8adf-c63c42d17453 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:53.399316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8f475-5edf-4d18-90a3-143e2f7119b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:53.401004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8f475-5edf-4d18-90a3-143e2f7119b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:53.401973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8f475-5edf-4d18-90a3-143e2f7119b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:53.403342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8f475-5edf-4d18-90a3-143e2f7119b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:53.404978 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8f475-5edf-4d18-90a3-143e2f7119b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:53.408527 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d8f475-5edf-4d18-90a3-143e2f7119b5 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:53.447887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 923/1838] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:53.448895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646067cc-a91e-4ff8-8ef7-ad28d5a8b2f2 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:53.478335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646067cc-a91e-4ff8-8ef7-ad28d5a8b2f2 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:53.490303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86336e-30b6-4b0f-99dc-a1d39d08ea46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:53.496665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86336e-30b6-4b0f-99dc-a1d39d08ea46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:53.497832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86336e-30b6-4b0f-99dc-a1d39d08ea46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:53.504198 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86336e-30b6-4b0f-99dc-a1d39d08ea46 admin admin] PATH_INFO: `/v3/projects/bf77dfa2810342ff8f037a62c26ecda7/users/af2d20794bfe4dfd84e7c2222064ea1a/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:53.553816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86336e-30b6-4b0f-99dc-a1d39d08ea46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf77dfa2810342ff8f037a62c26ecda7, user_id=af2d20794bfe4dfd84e7c2222064ea1a, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:53.561801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86336e-30b6-4b0f-99dc-a1d39d08ea46 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:53.665032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 916/1839] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:53 2022] PUT /identity/v3/projects/bf77dfa2810342ff8f037a62c26ecda7/users/af2d20794bfe4dfd84e7c2222064ea1a/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:53.686837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdccec4-3980-4281-b139-869c9b5024f1 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:53.687691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdccec4-3980-4281-b139-869c9b5024f1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:53.688414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdccec4-3980-4281-b139-869c9b5024f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:53.775847 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3cdccec4-3980-4281-b139-869c9b5024f1 None None] MFA Rules not processed for user `af2d20794bfe4dfd84e7c2222064ea1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:53.778833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3cdccec4-3980-4281-b139-869c9b5024f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:53.963903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 924/1840] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:53.969784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86336e-30b6-4b0f-99dc-a1d39d08ea46 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:54.140243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdccec4-3980-4281-b139-869c9b5024f1 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:54.142655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3cdccec4-3980-4281-b139-869c9b5024f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:54.180153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86336e-30b6-4b0f-99dc-a1d39d08ea46 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:54.193852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdccec4-3980-4281-b139-869c9b5024f1 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:54.209042 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623484e-3921-4bc3-a109-4b8f137c9b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:54.210106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623484e-3921-4bc3-a109-4b8f137c9b45 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:54.210576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623484e-3921-4bc3-a109-4b8f137c9b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:54.211044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623484e-3921-4bc3-a109-4b8f137c9b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:54.212237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623484e-3921-4bc3-a109-4b8f137c9b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:54.220299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623484e-3921-4bc3-a109-4b8f137c9b45 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:54.220970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:54.222076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:54.222610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:54.223094 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] PATH_INFO: `/v3/projects/573ed517b2c549ba95a552153718ba99` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:54.230515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=573ed517b2c549ba95a552153718ba99)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:54.240787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:54.413488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 917/1841] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:30:53 2022] POST /identity/v3/projects => generated 363 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:30:54.421407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623484e-3921-4bc3-a109-4b8f137c9b45 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:54.642948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623484e-3921-4bc3-a109-4b8f137c9b45 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:54.652894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573ed517b2c549ba95a552153718ba99', 'request_id': 'req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:54.686107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573ed517b2c549ba95a552153718ba99', 'request_id': 'req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:54.871287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09271311-e18c-40a1-bca9-1e2ca5ff3e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:54.873159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09271311-e18c-40a1-bca9-1e2ca5ff3e59 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:54.873965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09271311-e18c-40a1-bca9-1e2ca5ff3e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:54.874483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09271311-e18c-40a1-bca9-1e2ca5ff3e59 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:54.876409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09271311-e18c-40a1-bca9-1e2ca5ff3e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:54.881072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09271311-e18c-40a1-bca9-1e2ca5ff3e59 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:54.919635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573ed517b2c549ba95a552153718ba99', 'request_id': 'req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:54.996651 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] The token cache is being invalidate because project 573ed517b2c549ba95a552153718ba99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:30:54.997187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:54.998821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:55.003130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 925/1842] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:30:54 2022] DELETE /identity/v3/projects/573ed517b2c549ba95a552153718ba99 => generated 0 bytes in 865 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:55.013315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:55.013315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:55.063595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2f551-8f10-4ca1-899b-2c61177dc7e6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:55.085892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:55.087140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:55.089689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:55.091325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] PATH_INFO: `/v3/users/170ecfc06f4841c7ac2d754fa100a720` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:55.098546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170ecfc06f4841c7ac2d754fa100a720)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:55.107599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 918/1843] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:30:54 2022] POST /identity/v3/users => generated 288 bytes in 685 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:30:55.107923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:55.111528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09271311-e18c-40a1-bca9-1e2ca5ff3e59 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:55.116430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-09271311-e18c-40a1-bca9-1e2ca5ff3e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:55.174294 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09271311-e18c-40a1-bca9-1e2ca5ff3e59 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:55.204253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfc6e1-ae8a-402a-b57b-b5b0c7c386bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:55.208615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfc6e1-ae8a-402a-b57b-b5b0c7c386bb admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:55.209625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfc6e1-ae8a-402a-b57b-b5b0c7c386bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:55.211689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfc6e1-ae8a-402a-b57b-b5b0c7c386bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:55.213962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfc6e1-ae8a-402a-b57b-b5b0c7c386bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:55.223589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfc6e1-ae8a-402a-b57b-b5b0c7c386bb admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:55.273704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 919/1844] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:30:55.286540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfc6e1-ae8a-402a-b57b-b5b0c7c386bb admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:55.316938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfc6e1-ae8a-402a-b57b-b5b0c7c386bb admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:55.350732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d02ef5-4b70-482e-85a3-4eec136c0d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:55.354855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02ef5-4b70-482e-85a3-4eec136c0d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:55.356213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02ef5-4b70-482e-85a3-4eec136c0d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:55.360352 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02ef5-4b70-482e-85a3-4eec136c0d8a admin admin] PATH_INFO: `/v3/projects/19a28f6bc31146e9bb8225b22fdad682/users/15fd5b7106a4438ea55aa89743d137ed/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:55.446357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d02ef5-4b70-482e-85a3-4eec136c0d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a28f6bc31146e9bb8225b22fdad682, user_id=15fd5b7106a4438ea55aa89743d137ed, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:55.452997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d02ef5-4b70-482e-85a3-4eec136c0d8a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:55.473607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170ecfc06f4841c7ac2d754fa100a720'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:55.478020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170ecfc06f4841c7ac2d754fa100a720'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:55.479465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170ecfc06f4841c7ac2d754fa100a720'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:55.510946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170ecfc06f4841c7ac2d754fa100a720'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:55.585406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:55.600710 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 920/1845] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:55 2022] PUT /identity/v3/projects/19a28f6bc31146e9bb8225b22fdad682/users/15fd5b7106a4438ea55aa89743d137ed/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:30:55.607504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170ecfc06f4841c7ac2d754fa100a720'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:55.608543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d02ef5-4b70-482e-85a3-4eec136c0d8a admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:55.616743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-16d02ef5-4b70-482e-85a3-4eec136c0d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:55.799911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d02ef5-4b70-482e-85a3-4eec136c0d8a admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:55.809451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0f909-f0b6-451d-a16e-837ccfc7900f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:55.810542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0f909-f0b6-451d-a16e-837ccfc7900f admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:55.812972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0f909-f0b6-451d-a16e-837ccfc7900f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:55.813596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0f909-f0b6-451d-a16e-837ccfc7900f admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:55.814856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0f909-f0b6-451d-a16e-837ccfc7900f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:55.828439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0f909-f0b6-451d-a16e-837ccfc7900f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:55.834240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 926/1846] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:30:55 2022] DELETE /identity/v3/users/170ecfc06f4841c7ac2d754fa100a720 => generated 0 bytes in 825 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:55.864046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 921/1847] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:30:55.870089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:56.002420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034679c2-f69d-43fa-8390-3b123e10b0d8 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:56.009010 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad24af-b89e-423e-9fb7-a1fa9f63bb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:56.009798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad24af-b89e-423e-9fb7-a1fa9f63bb2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:56.010142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad24af-b89e-423e-9fb7-a1fa9f63bb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:56.010513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad24af-b89e-423e-9fb7-a1fa9f63bb2c admin admin] PATH_INFO: `/v3/projects/19a28f6bc31146e9bb8225b22fdad682/users/15fd5b7106a4438ea55aa89743d137ed/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:56.012984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad24af-b89e-423e-9fb7-a1fa9f63bb2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a28f6bc31146e9bb8225b22fdad682, user_id=15fd5b7106a4438ea55aa89743d137ed, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:56.016777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad24af-b89e-423e-9fb7-a1fa9f63bb2c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:56.037301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 927/1848] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:30:55 2022] PUT /identity/v3/projects/19a28f6bc31146e9bb8225b22fdad682/users/15fd5b7106a4438ea55aa89743d137ed/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:56.056624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09dafc-a791-4373-9946-1b1c5956432d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:56.057235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09dafc-a791-4373-9946-1b1c5956432d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:56.057697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba09dafc-a791-4373-9946-1b1c5956432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:56.100895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad24af-b89e-423e-9fb7-a1fa9f63bb2c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:56.108832 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-52ad24af-b89e-423e-9fb7-a1fa9f63bb2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:56.139858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ba09dafc-a791-4373-9946-1b1c5956432d None None] MFA Rules not processed for user `15fd5b7106a4438ea55aa89743d137ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:56.142749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ba09dafc-a791-4373-9946-1b1c5956432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:56.308938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad24af-b89e-423e-9fb7-a1fa9f63bb2c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:56.328634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:56.329711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:56.330033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:56.330423 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] PATH_INFO: `/v3/users/464fd0776f9942d88c9e3eb060721666` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:56.331872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 922/1849] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:56.332728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=464fd0776f9942d88c9e3eb060721666)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:56.333663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09dafc-a791-4373-9946-1b1c5956432d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:56.337045 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ba09dafc-a791-4373-9946-1b1c5956432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:56.338181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:56.371400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba09dafc-a791-4373-9946-1b1c5956432d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:56.384805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:56.386035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:56.386794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:56.387539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] PATH_INFO: `/v3/users/4076290ee0f8456b892918970f93b1bc` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:56.388278 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:56.391408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4076290ee0f8456b892918970f93b1bc)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:56.396561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:56.752629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '464fd0776f9942d88c9e3eb060721666'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:56.753450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '464fd0776f9942d88c9e3eb060721666'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:56.754234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '464fd0776f9942d88c9e3eb060721666'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:56.768608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4076290ee0f8456b892918970f93b1bc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:56.773528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '464fd0776f9942d88c9e3eb060721666'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:56.810825 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-af1615ad-b868-427e-bab4-8bd3dfa2de15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '464fd0776f9942d88c9e3eb060721666'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:56.942563 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4076290ee0f8456b892918970f93b1bc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:56.943705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4076290ee0f8456b892918970f93b1bc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:56.988742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4076290ee0f8456b892918970f93b1bc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:56.990113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4076290ee0f8456b892918970f93b1bc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:57.044373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 923/1850] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:30:56 2022] DELETE /identity/v3/users/4076290ee0f8456b892918970f93b1bc => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:57.047050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:57.049798 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:57.130551 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 928/1851] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:30:56 2022] DELETE /identity/v3/users/464fd0776f9942d88c9e3eb060721666 => generated 0 bytes in 1031 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:57.139860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180b0c6-1fe1-413a-82a0-deed721411c9 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:57.151666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180b0c6-1fe1-413a-82a0-deed721411c9 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:57.152319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a180b0c6-1fe1-413a-82a0-deed721411c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:57.225702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66019423-c23b-4702-9010-17a0158a9103 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:57.227838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a180b0c6-1fe1-413a-82a0-deed721411c9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:57.230755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a180b0c6-1fe1-413a-82a0-deed721411c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:57.336070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8380b-a82e-4f8f-bea6-3cca81b02bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:57.337014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8380b-a82e-4f8f-bea6-3cca81b02bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:57.337406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8380b-a82e-4f8f-bea6-3cca81b02bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:57.337876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8380b-a82e-4f8f-bea6-3cca81b02bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:57.342536 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f4c8380b-a82e-4f8f-bea6-3cca81b02bd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:57.369668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 929/1852] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:57.369668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42d379-bd47-40db-b7cd-c285a230f312 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:57.372046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42d379-bd47-40db-b7cd-c285a230f312 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:57.372424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42d379-bd47-40db-b7cd-c285a230f312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:57.401449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6f42d379-bd47-40db-b7cd-c285a230f312 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:30:57.402860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6f42d379-bd47-40db-b7cd-c285a230f312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:57.414294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 930/1853] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:30:57.415328 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42d379-bd47-40db-b7cd-c285a230f312 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:57.417416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6f42d379-bd47-40db-b7cd-c285a230f312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:57.437209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42d379-bd47-40db-b7cd-c285a230f312 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:57.444006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:57.444817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:57.445157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:57.445454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] PATH_INFO: `/v3/projects/dd5ae862be384f3e923dcb7041ae0b95` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:57.447741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd5ae862be384f3e923dcb7041ae0b95)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:57.451101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:57.458781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8380b-a82e-4f8f-bea6-3cca81b02bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:57.461432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8380b-a82e-4f8f-bea6-3cca81b02bd6 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:57.476151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 924/1854] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:57.477459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8380b-a82e-4f8f-bea6-3cca81b02bd6 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:57.479116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f4c8380b-a82e-4f8f-bea6-3cca81b02bd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:57.582366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8380b-a82e-4f8f-bea6-3cca81b02bd6 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:57.589007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18814c-e5a5-4c31-b0fb-29594c058d1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:57.590078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18814c-e5a5-4c31-b0fb-29594c058d1e service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:57.590817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18814c-e5a5-4c31-b0fb-29594c058d1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:57.591302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18814c-e5a5-4c31-b0fb-29594c058d1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:57.611139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18814c-e5a5-4c31-b0fb-29594c058d1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:57.612870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18814c-e5a5-4c31-b0fb-29594c058d1e service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:57.626675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 925/1855] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:57.627584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18814c-e5a5-4c31-b0fb-29594c058d1e service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:57.644954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18814c-e5a5-4c31-b0fb-29594c058d1e service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:57.649745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547ae30-f000-4f0c-8470-52bb293c77e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:57.650805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547ae30-f000-4f0c-8470-52bb293c77e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:57.651487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547ae30-f000-4f0c-8470-52bb293c77e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:57.652031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5547ae30-f000-4f0c-8470-52bb293c77e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:57.682507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd5ae862be384f3e923dcb7041ae0b95', 'request_id': 'req-511c38e1-2038-40d8-9c2f-590a6ffbc38f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:57.688791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547ae30-f000-4f0c-8470-52bb293c77e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:57.690399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5547ae30-f000-4f0c-8470-52bb293c77e6 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:57.699720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd5ae862be384f3e923dcb7041ae0b95', 'request_id': 'req-511c38e1-2038-40d8-9c2f-590a6ffbc38f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:57.719791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 926/1856] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:30:57.853331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd5ae862be384f3e923dcb7041ae0b95', 'request_id': 'req-511c38e1-2038-40d8-9c2f-590a6ffbc38f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:57.889891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] The token cache is being invalidate because project dd5ae862be384f3e923dcb7041ae0b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:30:57.890739 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:57.892085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:57.896607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 931/1857] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:30:57 2022] DELETE /identity/v3/projects/dd5ae862be384f3e923dcb7041ae0b95 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:57.903055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547ae30-f000-4f0c-8470-52bb293c77e6 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:57.925412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5547ae30-f000-4f0c-8470-52bb293c77e6 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:58.068772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:58.070197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:58.070834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:58.071505 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] PATH_INFO: `/v3/users/22f8068c20d24468870b08517bbb8cf7` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:58.074553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22f8068c20d24468870b08517bbb8cf7)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:58.079938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:58.716968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:58.719363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:58.746134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22f8068c20d24468870b08517bbb8cf7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:58.781055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c38e1-2038-40d8-9c2f-590a6ffbc38f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:58.793597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:58.794655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:58.795392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:58.795974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] PATH_INFO: `/v3/projects/d50e1d80c8844f589c1798fa194f3bb2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:58.798422 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d50e1d80c8844f589c1798fa194f3bb2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:58.820387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:58.995111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22f8068c20d24468870b08517bbb8cf7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:58.996234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22f8068c20d24468870b08517bbb8cf7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:59.001986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:30:59.030276 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22f8068c20d24468870b08517bbb8cf7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:59.034778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22f8068c20d24468870b08517bbb8cf7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:59.074317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 927/1858] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:30:57 2022] DELETE /identity/v3/users/22f8068c20d24468870b08517bbb8cf7 => generated 0 bytes in 1165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:59.076204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:59.079394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:59.172665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd50e1d80c8844f589c1798fa194f3bb2', 'request_id': 'req-6cf8c349-56af-41a6-bc28-5e16a7f06876'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:59.202711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd50e1d80c8844f589c1798fa194f3bb2', 'request_id': 'req-6cf8c349-56af-41a6-bc28-5e16a7f06876'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:59.273015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91314b85-657d-4bda-b3be-622a08403a91 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:59.342112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd50e1d80c8844f589c1798fa194f3bb2', 'request_id': 'req-6cf8c349-56af-41a6-bc28-5e16a7f06876'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:59.361533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] The token cache is being invalidate because project d50e1d80c8844f589c1798fa194f3bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:30:59.361894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:59.365508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:59.370118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 932/1859] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:30:58 2022] DELETE /identity/v3/projects/d50e1d80c8844f589c1798fa194f3bb2 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:30:59.385704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:30:59.388843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:30:59.399392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:59.400281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:59.400645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:59.400998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] PATH_INFO: `/v3/projects/7d433599ff464cd7913949db00685313` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:59.406341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d433599ff464cd7913949db00685313)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:59.410599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:59.432420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8c349-56af-41a6-bc28-5e16a7f06876 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:30:59.448124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:30:59.449433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:30:59.450690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:30:59.451290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] PATH_INFO: `/v3/users/54cdf776a23a435f8a71daa39e4beaad` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:30:59.457619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] RBAC: Authorizing `identity:delete_user(user_id=54cdf776a23a435f8a71daa39e4beaad)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:30:59.464577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:30:59.745394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d433599ff464cd7913949db00685313', 'request_id': 'req-ad52a6eb-934c-4161-ad69-8acd033f9122'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:30:59.745394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d433599ff464cd7913949db00685313', 'request_id': 'req-ad52a6eb-934c-4161-ad69-8acd033f9122'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.257223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d433599ff464cd7913949db00685313', 'request_id': 'req-ad52a6eb-934c-4161-ad69-8acd033f9122'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.280308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] The token cache is being invalidate because project 7d433599ff464cd7913949db00685313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:00.280842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.282032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.285215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 928/1860] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:30:59 2022] DELETE /identity/v3/projects/7d433599ff464cd7913949db00685313 => generated 0 bytes in 1210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:00.286380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:00.291737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:00.398853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54cdf776a23a435f8a71daa39e4beaad'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.398853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54cdf776a23a435f8a71daa39e4beaad'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.398853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54cdf776a23a435f8a71daa39e4beaad'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.461342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54cdf776a23a435f8a71daa39e4beaad'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.468087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52a6eb-934c-4161-ad69-8acd033f9122 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:00.473741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:00.476111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:00.476711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:00.477221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] PATH_INFO: `/v3/users/af2d20794bfe4dfd84e7c2222064ea1a` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:00.478876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2d20794bfe4dfd84e7c2222064ea1a)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:00.481438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:00.502750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54cdf776a23a435f8a71daa39e4beaad'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.675231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 933/1861] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:30:59 2022] DELETE /identity/v3/users/54cdf776a23a435f8a71daa39e4beaad => generated 0 bytes in 1296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:00.677818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:00.682226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:00.814944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2d20794bfe4dfd84e7c2222064ea1a'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.888577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af9342-78a6-4e79-beac-9e46b360ecee admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:00.901225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:00.909266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:00.909266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:00.909266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] PATH_INFO: `/v3/users/e559326dd25b4152b30be95ac2e0b944` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:00.909266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e559326dd25b4152b30be95ac2e0b944)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:00.910691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:00.942067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2d20794bfe4dfd84e7c2222064ea1a'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.943110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2d20794bfe4dfd84e7c2222064ea1a'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:00.997177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2d20794bfe4dfd84e7c2222064ea1a'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.001035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2d20794bfe4dfd84e7c2222064ea1a'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.030978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 929/1862] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:00 2022] DELETE /identity/v3/users/af2d20794bfe4dfd84e7c2222064ea1a => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:01.032615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:01.035292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:01.155758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87403c-6103-4c9d-8e46-cf4c2edb2809 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:01.164698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:01.165338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:01.165560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:01.165838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] PATH_INFO: `/v3/projects/9e3321b43dd64b4c88d2beefc5137c6a` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:01.167150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e3321b43dd64b4c88d2beefc5137c6a)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:01.169241 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:01.307591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e559326dd25b4152b30be95ac2e0b944'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.309538 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e559326dd25b4152b30be95ac2e0b944'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.310716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e559326dd25b4152b30be95ac2e0b944'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.324136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e559326dd25b4152b30be95ac2e0b944'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.348378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f8562dca-5149-4aa5-9cb1-96bed0941403 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e559326dd25b4152b30be95ac2e0b944'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.455009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e3321b43dd64b4c88d2beefc5137c6a', 'request_id': 'req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.464873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e3321b43dd64b4c88d2beefc5137c6a', 'request_id': 'req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.664026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 934/1863] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:00 2022] DELETE /identity/v3/users/e559326dd25b4152b30be95ac2e0b944 => generated 0 bytes in 988 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:01.669529 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba12fc9-c2d8-44bd-9626-e5fc6970ea72 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:01.671435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba12fc9-c2d8-44bd-9626-e5fc6970ea72 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:01.671943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba12fc9-c2d8-44bd-9626-e5fc6970ea72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:01.699682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:01.730127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-0ba12fc9-c2d8-44bd-9626-e5fc6970ea72 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:01.731530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0ba12fc9-c2d8-44bd-9626-e5fc6970ea72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:01.863691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 935/1864] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:01.864752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba12fc9-c2d8-44bd-9626-e5fc6970ea72 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:01.868786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0ba12fc9-c2d8-44bd-9626-e5fc6970ea72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:01.869974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e3321b43dd64b4c88d2beefc5137c6a', 'request_id': 'req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.884798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba12fc9-c2d8-44bd-9626-e5fc6970ea72 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:01.891725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:01.892790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:01.893517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:01.894237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] PATH_INFO: `/v3/projects/01bf2bb45236430d98a04cb5b30d423d` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:01.896123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01bf2bb45236430d98a04cb5b30d423d)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:01.900894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:01.920863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] The token cache is being invalidate because project 9e3321b43dd64b4c88d2beefc5137c6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:01.921349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.922696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:01.937551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 930/1865] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:01 2022] DELETE /identity/v3/projects/9e3321b43dd64b4c88d2beefc5137c6a => generated 0 bytes in 897 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:01.940232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:01.943264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:02.086916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9aae57-cef5-43f2-9853-d4c3bd9ab5a5 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:02.095533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01bf2bb45236430d98a04cb5b30d423d', 'request_id': 'req-1f1455a4-b3bb-4b95-854a-237a818075a2'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:02.106871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01bf2bb45236430d98a04cb5b30d423d', 'request_id': 'req-1f1455a4-b3bb-4b95-854a-237a818075a2'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:02.184747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a564068-28e6-4357-9007-874956ae8aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:02.186446 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a564068-28e6-4357-9007-874956ae8aaf service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:02.186863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a564068-28e6-4357-9007-874956ae8aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:02.187264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a564068-28e6-4357-9007-874956ae8aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:02.190135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6a564068-28e6-4357-9007-874956ae8aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:02.193893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01bf2bb45236430d98a04cb5b30d423d', 'request_id': 'req-1f1455a4-b3bb-4b95-854a-237a818075a2'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:02.238458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] The token cache is being invalidate because project 01bf2bb45236430d98a04cb5b30d423d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:02.239457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:02.241129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1f1455a4-b3bb-4b95-854a-237a818075a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:02.244372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 936/1866] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:01 2022] DELETE /identity/v3/projects/01bf2bb45236430d98a04cb5b30d423d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:02.247421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50505c-e276-412c-a71f-c86c2180f52a None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:02.249348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50505c-e276-412c-a71f-c86c2180f52a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:02.250086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50505c-e276-412c-a71f-c86c2180f52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:02.294927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-4d50505c-e276-412c-a71f-c86c2180f52a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:02.299909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4d50505c-e276-412c-a71f-c86c2180f52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:02.412279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a564068-28e6-4357-9007-874956ae8aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:02.419250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a564068-28e6-4357-9007-874956ae8aaf service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:02.446454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 931/1867] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:02.450079 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:02.457755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044fafb-12bf-442e-b6d6-20f57761d7a8 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:02.457755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044fafb-12bf-442e-b6d6-20f57761d7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:02.458481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044fafb-12bf-442e-b6d6-20f57761d7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:02.461960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 937/1868] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:02.471298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee97c0-6ea7-4850-9ac7-c5c8b9a45124 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:02.471925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee97c0-6ea7-4850-9ac7-c5c8b9a45124 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:02.472240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee97c0-6ea7-4850-9ac7-c5c8b9a45124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:02.501254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-47ee97c0-6ea7-4850-9ac7-c5c8b9a45124 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:02.506252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-47ee97c0-6ea7-4850-9ac7-c5c8b9a45124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:02.535645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-1044fafb-12bf-442e-b6d6-20f57761d7a8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:02.539300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 938/1869] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:02.540799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1044fafb-12bf-442e-b6d6-20f57761d7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:02.548546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee97c0-6ea7-4850-9ac7-c5c8b9a45124 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:02.569803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee97c0-6ea7-4850-9ac7-c5c8b9a45124 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:02.587700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 932/1870] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:02.594720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe824f-a363-4b1e-bb2e-5b67c7bf6a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:02.594720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe824f-a363-4b1e-bb2e-5b67c7bf6a30 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:02.594720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe824f-a363-4b1e-bb2e-5b67c7bf6a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:02.594720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe824f-a363-4b1e-bb2e-5b67c7bf6a30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:02.595276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe824f-a363-4b1e-bb2e-5b67c7bf6a30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:02.595412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe824f-a363-4b1e-bb2e-5b67c7bf6a30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:02.595587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b687e8e-1574-490c-8f6c-9e5e8b683ab3 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:02.596228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b687e8e-1574-490c-8f6c-9e5e8b683ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:02.596228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b687e8e-1574-490c-8f6c-9e5e8b683ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:02.599347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe824f-a363-4b1e-bb2e-5b67c7bf6a30 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:02.644845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-6b687e8e-1574-490c-8f6c-9e5e8b683ab3 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:02.645054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6b687e8e-1574-490c-8f6c-9e5e8b683ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:02.652345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 939/1871] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:02.654092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe824f-a363-4b1e-bb2e-5b67c7bf6a30 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:02.668151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 933/1872] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:02.674839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b687e8e-1574-490c-8f6c-9e5e8b683ab3 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:02.679845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe824f-a363-4b1e-bb2e-5b67c7bf6a30 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:02.693567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b687e8e-1574-490c-8f6c-9e5e8b683ab3 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:02.699214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e0f74-4cdd-4e4c-a2a6-981c6c1828f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:02.700914 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e0f74-4cdd-4e4c-a2a6-981c6c1828f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:02.701477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e0f74-4cdd-4e4c-a2a6-981c6c1828f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:02.703831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e0f74-4cdd-4e4c-a2a6-981c6c1828f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:02.705557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e0f74-4cdd-4e4c-a2a6-981c6c1828f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:02.707006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29938e7b-658d-4fc1-9182-33ef8bc2d486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:02.708440 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29938e7b-658d-4fc1-9182-33ef8bc2d486 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:02.713210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29938e7b-658d-4fc1-9182-33ef8bc2d486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:02.713358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e0f74-4cdd-4e4c-a2a6-981c6c1828f7 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:02.713450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29938e7b-658d-4fc1-9182-33ef8bc2d486 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:02.713450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29938e7b-658d-4fc1-9182-33ef8bc2d486 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:02.713450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29938e7b-658d-4fc1-9182-33ef8bc2d486 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:02.715233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29938e7b-658d-4fc1-9182-33ef8bc2d486 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:02.747136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 934/1873] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:02.755693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29938e7b-658d-4fc1-9182-33ef8bc2d486 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:02.772312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29938e7b-658d-4fc1-9182-33ef8bc2d486 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:02.779518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f0c63-1238-4b23-b508-68575d43d402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:02.782827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f0c63-1238-4b23-b508-68575d43d402 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:02.782827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f0c63-1238-4b23-b508-68575d43d402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:02.782827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f0c63-1238-4b23-b508-68575d43d402 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:02.782827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f0c63-1238-4b23-b508-68575d43d402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:02.784171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f0c63-1238-4b23-b508-68575d43d402 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:02.886061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 940/1874] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:02 2022] POST /identity/v3/projects => generated 361 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:02.892600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e0f74-4cdd-4e4c-a2a6-981c6c1828f7 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:02.924148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 935/1875] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:02 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:02.926131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f0c63-1238-4b23-b508-68575d43d402 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:03.037792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e0f74-4cdd-4e4c-a2a6-981c6c1828f7 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:03.095026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f0c63-1238-4b23-b508-68575d43d402 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:03.106537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec32c1-6fe5-43a4-8e72-00443686bed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:03.108922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec32c1-6fe5-43a4-8e72-00443686bed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.109584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec32c1-6fe5-43a4-8e72-00443686bed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.110050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ec32c1-6fe5-43a4-8e72-00443686bed3 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.110965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec32c1-6fe5-43a4-8e72-00443686bed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:03.115457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ec32c1-6fe5-43a4-8e72-00443686bed3 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:03.137539 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9831d-2ad9-4a63-9b30-7883a59a39f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:03.139312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9831d-2ad9-4a63-9b30-7883a59a39f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.139919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9831d-2ad9-4a63-9b30-7883a59a39f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.144702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9831d-2ad9-4a63-9b30-7883a59a39f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.144702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9831d-2ad9-4a63-9b30-7883a59a39f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:03.146231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9831d-2ad9-4a63-9b30-7883a59a39f2 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:03.254749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 936/1876] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:02 2022] POST /identity/v3/users => generated 285 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:03.259531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec32c1-6fe5-43a4-8e72-00443686bed3 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:03.284753 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ec32c1-6fe5-43a4-8e72-00443686bed3 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:03.291482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 941/1877] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:02 2022] POST /identity/v3/users => generated 281 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:03.296231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9831d-2ad9-4a63-9b30-7883a59a39f2 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:03.299496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf3de-6b08-41bc-9d04-0fedd2b1cbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:03.300367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf3de-6b08-41bc-9d04-0fedd2b1cbfa admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.300762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf3de-6b08-41bc-9d04-0fedd2b1cbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.301231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fdf3de-6b08-41bc-9d04-0fedd2b1cbfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.307830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdf3de-6b08-41bc-9d04-0fedd2b1cbfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:03.311474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fdf3de-6b08-41bc-9d04-0fedd2b1cbfa admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:03.321260 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9831d-2ad9-4a63-9b30-7883a59a39f2 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:03.333441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cab6e0-0f2b-4447-a84a-a8c6107a8f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:03.335335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cab6e0-0f2b-4447-a84a-a8c6107a8f74 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.335859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cab6e0-0f2b-4447-a84a-a8c6107a8f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.336211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cab6e0-0f2b-4447-a84a-a8c6107a8f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.336918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cab6e0-0f2b-4447-a84a-a8c6107a8f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:03.340140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cab6e0-0f2b-4447-a84a-a8c6107a8f74 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:03.364242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 942/1879] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:03.365337 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cab6e0-0f2b-4447-a84a-a8c6107a8f74 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:03.372035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 937/1879] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:03.373219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf3de-6b08-41bc-9d04-0fedd2b1cbfa admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:03.380508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cab6e0-0f2b-4447-a84a-a8c6107a8f74 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:03.387381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db933f-fc5d-4e09-b20f-52dd4c63ecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:03.387639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fdf3de-6b08-41bc-9d04-0fedd2b1cbfa admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:03.388265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db933f-fc5d-4e09-b20f-52dd4c63ecd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.391044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db933f-fc5d-4e09-b20f-52dd4c63ecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.391044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db933f-fc5d-4e09-b20f-52dd4c63ecd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.391044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db933f-fc5d-4e09-b20f-52dd4c63ecd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:03.391930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db933f-fc5d-4e09-b20f-52dd4c63ecd9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:03.394233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e60b41-b35c-4918-b5f1-3df1399d4de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:03.395092 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e60b41-b35c-4918-b5f1-3df1399d4de2 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.395452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e60b41-b35c-4918-b5f1-3df1399d4de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.395863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e60b41-b35c-4918-b5f1-3df1399d4de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.396724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e60b41-b35c-4918-b5f1-3df1399d4de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:03.398534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e60b41-b35c-4918-b5f1-3df1399d4de2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:03.413175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 943/1880] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:03.419112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 938/1881] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:03.420617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db933f-fc5d-4e09-b20f-52dd4c63ecd9 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:03.425060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e60b41-b35c-4918-b5f1-3df1399d4de2 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:03.435602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db933f-fc5d-4e09-b20f-52dd4c63ecd9 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:03.440489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e60b41-b35c-4918-b5f1-3df1399d4de2 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:03.441745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f378cb-429f-44e4-bea8-ebee88b43d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:03.442577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f378cb-429f-44e4-bea8-ebee88b43d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.443006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f378cb-429f-44e4-bea8-ebee88b43d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.443349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f378cb-429f-44e4-bea8-ebee88b43d35 admin admin] PATH_INFO: `/v3/projects/c0c46cf4b4964719b2108816d34b30dd/users/f67f7b66a8b745db827c0f4d9db181f6/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.448815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc831cf6-e0b1-414a-b991-b9532dbd10fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:03.449825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc831cf6-e0b1-414a-b991-b9532dbd10fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.450108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc831cf6-e0b1-414a-b991-b9532dbd10fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.450464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc831cf6-e0b1-414a-b991-b9532dbd10fe admin admin] PATH_INFO: `/v3/projects/7f282725bba8457983a07f75e413da76/users/e371897751d84e38af0492be6212ce88/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.464462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f378cb-429f-44e4-bea8-ebee88b43d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0c46cf4b4964719b2108816d34b30dd, user_id=f67f7b66a8b745db827c0f4d9db181f6, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:03.469335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f378cb-429f-44e4-bea8-ebee88b43d35 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:03.481090 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc831cf6-e0b1-414a-b991-b9532dbd10fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f282725bba8457983a07f75e413da76, user_id=e371897751d84e38af0492be6212ce88, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:03.485193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc831cf6-e0b1-414a-b991-b9532dbd10fe admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:03.638254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 939/1882] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:03 2022] PUT /identity/v3/projects/7f282725bba8457983a07f75e413da76/users/e371897751d84e38af0492be6212ce88/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:03.641476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc831cf6-e0b1-414a-b991-b9532dbd10fe admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:03.650977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fc831cf6-e0b1-414a-b991-b9532dbd10fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:03.653535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 944/1883] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:03 2022] PUT /identity/v3/projects/c0c46cf4b4964719b2108816d34b30dd/users/f67f7b66a8b745db827c0f4d9db181f6/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:03.661363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc56ce0-f1fc-4fb0-9d5e-dd92b4c94526 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.661656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc56ce0-f1fc-4fb0-9d5e-dd92b4c94526 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.663295 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc56ce0-f1fc-4fb0-9d5e-dd92b4c94526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.712685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-1fc56ce0-f1fc-4fb0-9d5e-dd92b4c94526 None None] MFA Rules not processed for user `e371897751d84e38af0492be6212ce88`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:03.714028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1fc56ce0-f1fc-4fb0-9d5e-dd92b4c94526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:03.786431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc831cf6-e0b1-414a-b991-b9532dbd10fe admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:03.794607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:03.795636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.796070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.796535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] PATH_INFO: `/v3/projects/314224f2e7364d1f93c7143b45a7e065` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.797951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=314224f2e7364d1f93c7143b45a7e065)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:03.800129 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:03.804208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 945/1884] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:03.805745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b65714-2d5e-4295-9476-69b217aa966d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:03.806418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b65714-2d5e-4295-9476-69b217aa966d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:03.806418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b65714-2d5e-4295-9476-69b217aa966d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:03.861652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-e1b65714-2d5e-4295-9476-69b217aa966d None None] MFA Rules not processed for user `f67f7b66a8b745db827c0f4d9db181f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:03.864017 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e1b65714-2d5e-4295-9476-69b217aa966d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:03.994024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '314224f2e7364d1f93c7143b45a7e065', 'request_id': 'req-e3a40bfa-a989-49ed-9ab2-944908d4bc36'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.005531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '314224f2e7364d1f93c7143b45a7e065', 'request_id': 'req-e3a40bfa-a989-49ed-9ab2-944908d4bc36'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.017995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 946/1885] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:04.017995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b65714-2d5e-4295-9476-69b217aa966d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:04.017995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e1b65714-2d5e-4295-9476-69b217aa966d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:04.099801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b65714-2d5e-4295-9476-69b217aa966d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:04.120134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '314224f2e7364d1f93c7143b45a7e065', 'request_id': 'req-e3a40bfa-a989-49ed-9ab2-944908d4bc36'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.141790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] The token cache is being invalidate because project 314224f2e7364d1f93c7143b45a7e065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:04.142317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.143228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.145735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 940/1886] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:03 2022] DELETE /identity/v3/projects/314224f2e7364d1f93c7143b45a7e065 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:04.146499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:04.149342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:04.174087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9877660-e552-484c-818a-5e5ca1493f75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:04.174739 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9877660-e552-484c-818a-5e5ca1493f75 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:04.175027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9877660-e552-484c-818a-5e5ca1493f75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:04.175297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9877660-e552-484c-818a-5e5ca1493f75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:04.180632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b9877660-e552-484c-818a-5e5ca1493f75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:04.244359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a40bfa-a989-49ed-9ab2-944908d4bc36 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:04.250058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe714fd0-16bd-4fb5-ba04-15f28ffa9df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:04.250698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe714fd0-16bd-4fb5-ba04-15f28ffa9df8 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:04.250987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe714fd0-16bd-4fb5-ba04-15f28ffa9df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:04.251265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe714fd0-16bd-4fb5-ba04-15f28ffa9df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:04.251942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe714fd0-16bd-4fb5-ba04-15f28ffa9df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:04.254031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe714fd0-16bd-4fb5-ba04-15f28ffa9df8 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:04.255769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9877660-e552-484c-818a-5e5ca1493f75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:04.257430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9877660-e552-484c-818a-5e5ca1493f75 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:04.268603 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 947/1887] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:04.270170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a3055-9e0b-4c7a-b471-68b003ca0d7c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:04.270458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a3055-9e0b-4c7a-b471-68b003ca0d7c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:04.270918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a3055-9e0b-4c7a-b471-68b003ca0d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:04.310219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9a0a3055-9e0b-4c7a-b471-68b003ca0d7c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:04.312797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9a0a3055-9e0b-4c7a-b471-68b003ca0d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:04.323693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 948/1888] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:04.324967 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a3055-9e0b-4c7a-b471-68b003ca0d7c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:04.326771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9a0a3055-9e0b-4c7a-b471-68b003ca0d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:04.341105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a3055-9e0b-4c7a-b471-68b003ca0d7c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:04.347701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:04.348573 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:04.348860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:04.349143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] PATH_INFO: `/v3/users/e371897751d84e38af0492be6212ce88` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:04.350699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e371897751d84e38af0492be6212ce88)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:04.352929 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:04.382936 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 941/1889] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:04 2022] POST /identity/v3/projects => generated 369 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:04.384615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe714fd0-16bd-4fb5-ba04-15f28ffa9df8 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:04.388306 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fe714fd0-16bd-4fb5-ba04-15f28ffa9df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:04.415799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:04.526096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe714fd0-16bd-4fb5-ba04-15f28ffa9df8 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:04.532546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:04.533216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:04.533500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:04.534018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] PATH_INFO: `/v3/projects/bf77dfa2810342ff8f037a62c26ecda7` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:04.535185 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf77dfa2810342ff8f037a62c26ecda7)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:04.537452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:04.706627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e371897751d84e38af0492be6212ce88'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.707875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e371897751d84e38af0492be6212ce88'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.708797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e371897751d84e38af0492be6212ce88'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.722221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e371897751d84e38af0492be6212ce88'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.751865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-713abac3-a4b6-47e4-a2d1-17616f94cec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e371897751d84e38af0492be6212ce88'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.764497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf77dfa2810342ff8f037a62c26ecda7', 'request_id': 'req-f3fa2018-e304-4b83-b76a-d0391cfe8581'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.776644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf77dfa2810342ff8f037a62c26ecda7', 'request_id': 'req-f3fa2018-e304-4b83-b76a-d0391cfe8581'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.854445 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 949/1890] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:04 2022] DELETE /identity/v3/users/e371897751d84e38af0492be6212ce88 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:04.856353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521bfbc-53f0-4f59-9aea-557779258cff None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:04.856652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521bfbc-53f0-4f59-9aea-557779258cff None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:04.856974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521bfbc-53f0-4f59-9aea-557779258cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:04.887755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-e521bfbc-53f0-4f59-9aea-557779258cff None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:04.889747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e521bfbc-53f0-4f59-9aea-557779258cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:04.918679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf77dfa2810342ff8f037a62c26ecda7', 'request_id': 'req-f3fa2018-e304-4b83-b76a-d0391cfe8581'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.935409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] The token cache is being invalidate because project bf77dfa2810342ff8f037a62c26ecda7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:04.936558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.937432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:04.939963 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 942/1891] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:04 2022] DELETE /identity/v3/projects/bf77dfa2810342ff8f037a62c26ecda7 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:04.942641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:04.944241 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:04.995758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 950/1892] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:04.997556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ce2ec-3e04-4709-b9ff-2161d2b8c51d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:04.997712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:04.997847 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ce2ec-3e04-4709-b9ff-2161d2b8c51d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:04.998130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ce2ec-3e04-4709-b9ff-2161d2b8c51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:05.020784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3c7ce2ec-3e04-4709-b9ff-2161d2b8c51d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:05.022222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3c7ce2ec-3e04-4709-b9ff-2161d2b8c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:05.033056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 951/1893] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:05.034301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ce2ec-3e04-4709-b9ff-2161d2b8c51d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:05.036069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3c7ce2ec-3e04-4709-b9ff-2161d2b8c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:05.056002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ce2ec-3e04-4709-b9ff-2161d2b8c51d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:05.069966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:05.070643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:05.070964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:05.071349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] PATH_INFO: `/v3/users/15fd5b7106a4438ea55aa89743d137ed` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:05.073274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] RBAC: Authorizing `identity:delete_user(user_id=15fd5b7106a4438ea55aa89743d137ed)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:05.075451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:05.095802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa2018-e304-4b83-b76a-d0391cfe8581 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:05.104637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec1279-de1f-40a0-ab1f-2b85a3bbc003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:05.105803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec1279-de1f-40a0-ab1f-2b85a3bbc003 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:05.106301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec1279-de1f-40a0-ab1f-2b85a3bbc003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:05.106795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec1279-de1f-40a0-ab1f-2b85a3bbc003 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:05.108327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec1279-de1f-40a0-ab1f-2b85a3bbc003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:05.112305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ec1279-de1f-40a0-ab1f-2b85a3bbc003 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:05.229208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 943/1894] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:04 2022] POST /identity/v3/users => generated 291 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:05.230583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec1279-de1f-40a0-ab1f-2b85a3bbc003 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:05.250545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec1279-de1f-40a0-ab1f-2b85a3bbc003 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:05.264980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c53b0-5740-44b0-8214-f1b03af98dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:05.265777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c53b0-5740-44b0-8214-f1b03af98dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:05.266596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c53b0-5740-44b0-8214-f1b03af98dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:05.266596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c53b0-5740-44b0-8214-f1b03af98dfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:05.267779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c53b0-5740-44b0-8214-f1b03af98dfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:05.268329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c53b0-5740-44b0-8214-f1b03af98dfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:05.270510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c53b0-5740-44b0-8214-f1b03af98dfc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:05.301460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 944/1895] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:05.303475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c53b0-5740-44b0-8214-f1b03af98dfc admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:05.306754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-398c53b0-5740-44b0-8214-f1b03af98dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:05.376153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c53b0-5740-44b0-8214-f1b03af98dfc admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:05.380486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f3522-8561-4562-9d06-c652690e967b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:05.381210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f3522-8561-4562-9d06-c652690e967b service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:05.381569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f3522-8561-4562-9d06-c652690e967b service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:05.381968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f3522-8561-4562-9d06-c652690e967b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:05.402810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f3522-8561-4562-9d06-c652690e967b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:05.406303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f3522-8561-4562-9d06-c652690e967b service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:05.431144 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 945/1896] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:05.432417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f3522-8561-4562-9d06-c652690e967b service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:05.444542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f3522-8561-4562-9d06-c652690e967b service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:05.450894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5eafd4-7434-4721-b1d4-8bbbe94770f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:05.451605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5eafd4-7434-4721-b1d4-8bbbe94770f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:05.451977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5eafd4-7434-4721-b1d4-8bbbe94770f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:05.452299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5eafd4-7434-4721-b1d4-8bbbe94770f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:05.453007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5eafd4-7434-4721-b1d4-8bbbe94770f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:05.455619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5eafd4-7434-4721-b1d4-8bbbe94770f5 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:05.474030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 946/1897] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:05.479778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5eafd4-7434-4721-b1d4-8bbbe94770f5 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:05.500641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5eafd4-7434-4721-b1d4-8bbbe94770f5 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:05.509754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14928aff-2af2-4d94-b665-3c609f1f20a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:05.509754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14928aff-2af2-4d94-b665-3c609f1f20a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:05.510149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14928aff-2af2-4d94-b665-3c609f1f20a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:05.510149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14928aff-2af2-4d94-b665-3c609f1f20a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:05.510912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14928aff-2af2-4d94-b665-3c609f1f20a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:05.511116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15fd5b7106a4438ea55aa89743d137ed'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:05.512714 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15fd5b7106a4438ea55aa89743d137ed'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:05.513133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14928aff-2af2-4d94-b665-3c609f1f20a9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:05.513601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15fd5b7106a4438ea55aa89743d137ed'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:05.546957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15fd5b7106a4438ea55aa89743d137ed'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:05.571526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15fd5b7106a4438ea55aa89743d137ed'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:05.648494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 947/1898] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:05 2022] POST /identity/v3/projects => generated 379 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:05.650055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14928aff-2af2-4d94-b665-3c609f1f20a9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:05.653413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-14928aff-2af2-4d94-b665-3c609f1f20a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:05.668965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:05.679470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 952/1899] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:05 2022] DELETE /identity/v3/users/15fd5b7106a4438ea55aa89743d137ed => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:05.681243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:05.695766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:05.804870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14928aff-2af2-4d94-b665-3c609f1f20a9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:05.883679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8db9-2347-44c9-8c2b-7bdc50a0780a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:05.892843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144013b-f9d5-47c2-a32a-7e226cd11dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:05.893849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144013b-f9d5-47c2-a32a-7e226cd11dce admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:05.894463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144013b-f9d5-47c2-a32a-7e226cd11dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:05.895002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144013b-f9d5-47c2-a32a-7e226cd11dce admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:05.896431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144013b-f9d5-47c2-a32a-7e226cd11dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:05.900207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144013b-f9d5-47c2-a32a-7e226cd11dce admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:05.949546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc3fa3-f602-4048-aa9b-f860f8cf068d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:05.950409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc3fa3-f602-4048-aa9b-f860f8cf068d admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:05.950806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc3fa3-f602-4048-aa9b-f860f8cf068d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:05.951099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc3fa3-f602-4048-aa9b-f860f8cf068d admin admin] PATH_INFO: `/v3/projects/d0288b9bb60049c2bc2d4732eed44f78/users/d6cb65cb9d914b8a80296e9311ef1212/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:05.981085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc3fa3-f602-4048-aa9b-f860f8cf068d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0288b9bb60049c2bc2d4732eed44f78, user_id=d6cb65cb9d914b8a80296e9311ef1212, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:05.987882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc3fa3-f602-4048-aa9b-f860f8cf068d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:06.009090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 953/1900] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:05 2022] POST /identity/v3/users => generated 290 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:06.013799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144013b-f9d5-47c2-a32a-7e226cd11dce admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:06.033011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144013b-f9d5-47c2-a32a-7e226cd11dce admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:06.073177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a43a6-ac64-4447-9f71-eeb68da7da5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:06.077640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a43a6-ac64-4447-9f71-eeb68da7da5a admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:06.088821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a43a6-ac64-4447-9f71-eeb68da7da5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:06.088821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a43a6-ac64-4447-9f71-eeb68da7da5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:06.088821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a43a6-ac64-4447-9f71-eeb68da7da5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:06.088821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a43a6-ac64-4447-9f71-eeb68da7da5a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:06.115644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 954/1901] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:06.122314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a43a6-ac64-4447-9f71-eeb68da7da5a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:06.165104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 948/1902] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:05 2022] PUT /identity/v3/projects/d0288b9bb60049c2bc2d4732eed44f78/users/d6cb65cb9d914b8a80296e9311ef1212/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:06.167302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a43a6-ac64-4447-9f71-eeb68da7da5a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:06.169819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc3fa3-f602-4048-aa9b-f860f8cf068d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:06.295934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e34cd8-eb72-467f-9b7c-ebecf67ebbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:06.296891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e34cd8-eb72-467f-9b7c-ebecf67ebbff admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:06.297092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e34cd8-eb72-467f-9b7c-ebecf67ebbff admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:06.297438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e34cd8-eb72-467f-9b7c-ebecf67ebbff admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:06.298135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e34cd8-eb72-467f-9b7c-ebecf67ebbff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:06.300594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e34cd8-eb72-467f-9b7c-ebecf67ebbff admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:06.313676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 955/1903] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:06.318987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e34cd8-eb72-467f-9b7c-ebecf67ebbff admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:06.361703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e34cd8-eb72-467f-9b7c-ebecf67ebbff admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:06.379433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d5192-d6b2-41ee-a84f-50311d5e3932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:06.380325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5192-d6b2-41ee-a84f-50311d5e3932 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:06.380611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5192-d6b2-41ee-a84f-50311d5e3932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:06.380888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5192-d6b2-41ee-a84f-50311d5e3932 admin admin] PATH_INFO: `/v3/projects/bf7ce4be9dd040f7bbe58e1c93ed5881/users/e41b7b6f256c422299984fdd1fffe7bb/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:06.383574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc3fa3-f602-4048-aa9b-f860f8cf068d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:06.398654 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf56898-2338-447d-b81d-a40263db8951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:06.400127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf56898-2338-447d-b81d-a40263db8951 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:06.400413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf56898-2338-447d-b81d-a40263db8951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:06.400688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf56898-2338-447d-b81d-a40263db8951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:06.401375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf56898-2338-447d-b81d-a40263db8951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:06.405793 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf56898-2338-447d-b81d-a40263db8951 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:06.424740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d5192-d6b2-41ee-a84f-50311d5e3932 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf7ce4be9dd040f7bbe58e1c93ed5881, user_id=e41b7b6f256c422299984fdd1fffe7bb, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:06.428001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d5192-d6b2-41ee-a84f-50311d5e3932 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:06.434586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 949/1904] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:06.440872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf56898-2338-447d-b81d-a40263db8951 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:06.472495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf56898-2338-447d-b81d-a40263db8951 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:06.488077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df96ee5-d296-456b-ab9e-6b93a2a7e453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:06.488077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df96ee5-d296-456b-ab9e-6b93a2a7e453 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:06.488077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df96ee5-d296-456b-ab9e-6b93a2a7e453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:06.488077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df96ee5-d296-456b-ab9e-6b93a2a7e453 admin admin] PATH_INFO: `/v3/projects/d0288b9bb60049c2bc2d4732eed44f78/users/d6cb65cb9d914b8a80296e9311ef1212/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:06.488699 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df96ee5-d296-456b-ab9e-6b93a2a7e453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0288b9bb60049c2bc2d4732eed44f78, user_id=d6cb65cb9d914b8a80296e9311ef1212, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:06.491057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df96ee5-d296-456b-ab9e-6b93a2a7e453 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:06.516886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 950/1905] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:06 2022] PUT /identity/v3/projects/d0288b9bb60049c2bc2d4732eed44f78/users/d6cb65cb9d914b8a80296e9311ef1212/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:06.537250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31683e3-682c-4b82-96be-a1b678ad70a5 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:06.537518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31683e3-682c-4b82-96be-a1b678ad70a5 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:06.537805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31683e3-682c-4b82-96be-a1b678ad70a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:06.579025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e31683e3-682c-4b82-96be-a1b678ad70a5 None None] MFA Rules not processed for user `d6cb65cb9d914b8a80296e9311ef1212`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:06.580482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e31683e3-682c-4b82-96be-a1b678ad70a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:06.620498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 956/1906] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:06 2022] PUT /identity/v3/projects/bf7ce4be9dd040f7bbe58e1c93ed5881/users/e41b7b6f256c422299984fdd1fffe7bb/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:06.634550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7840403-c893-4cca-b824-f0f20cdcc8e5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:06.636320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7840403-c893-4cca-b824-f0f20cdcc8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:06.637095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7840403-c893-4cca-b824-f0f20cdcc8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:06.689704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a7840403-c893-4cca-b824-f0f20cdcc8e5 None None] MFA Rules not processed for user `e41b7b6f256c422299984fdd1fffe7bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:06.691112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a7840403-c893-4cca-b824-f0f20cdcc8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:06.744989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:06.750018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 951/1907] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:06.750588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31683e3-682c-4b82-96be-a1b678ad70a5 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:06.753472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e31683e3-682c-4b82-96be-a1b678ad70a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:06.859063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 957/1908] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:06.859063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7840403-c893-4cca-b824-f0f20cdcc8e5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:06.859063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a7840403-c893-4cca-b824-f0f20cdcc8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:06.925739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31683e3-682c-4b82-96be-a1b678ad70a5 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:06.961972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7840403-c893-4cca-b824-f0f20cdcc8e5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:06.967618 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90067b23-f646-4db0-995e-080c560dbddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:06.968371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90067b23-f646-4db0-995e-080c560dbddc service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:06.968664 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90067b23-f646-4db0-995e-080c560dbddc service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:06.968940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90067b23-f646-4db0-995e-080c560dbddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:07.082346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:07.082914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:07.083197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:07.083476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] PATH_INFO: `/v3/projects/19a28f6bc31146e9bb8225b22fdad682` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:07.084788 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a28f6bc31146e9bb8225b22fdad682)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:07.087691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:07.112470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90067b23-f646-4db0-995e-080c560dbddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:07.114104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90067b23-f646-4db0-995e-080c560dbddc service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:07.137482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 958/1909] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:07.144888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90067b23-f646-4db0-995e-080c560dbddc service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:07.144888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-90067b23-f646-4db0-995e-080c560dbddc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:07.171473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90067b23-f646-4db0-995e-080c560dbddc service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:07.179759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:07.180784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:07.181263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:07.181736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] PATH_INFO: `/v3/projects/7f282725bba8457983a07f75e413da76` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:07.185267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f282725bba8457983a07f75e413da76)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:07.195194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:07.298870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a28f6bc31146e9bb8225b22fdad682', 'request_id': 'req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:07.310707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a28f6bc31146e9bb8225b22fdad682', 'request_id': 'req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:07.439508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a28f6bc31146e9bb8225b22fdad682', 'request_id': 'req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:07.441031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f282725bba8457983a07f75e413da76', 'request_id': 'req-25e2622f-0a56-4a1c-a341-2144c1f0893d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:07.454875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] The token cache is being invalidate because project 19a28f6bc31146e9bb8225b22fdad682 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:07.455522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:07.456609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f282725bba8457983a07f75e413da76', 'request_id': 'req-25e2622f-0a56-4a1c-a341-2144c1f0893d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:07.456841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:07.459660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 952/1910] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:06 2022] DELETE /identity/v3/projects/19a28f6bc31146e9bb8225b22fdad682 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:07.460919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:07.462556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:07.555969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae357c22-cf8e-4ef5-987e-24bc58cefa6b admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:07.566803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f282725bba8457983a07f75e413da76', 'request_id': 'req-25e2622f-0a56-4a1c-a341-2144c1f0893d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:07.589580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] The token cache is being invalidate because project 7f282725bba8457983a07f75e413da76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:07.590152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:07.592842 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-25e2622f-0a56-4a1c-a341-2144c1f0893d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:07.597218 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 959/1911] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:07 2022] DELETE /identity/v3/projects/7f282725bba8457983a07f75e413da76 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:07.600623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02c901-9a4e-4ceb-ba09-6a4ab7c0fed5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:07.601300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02c901-9a4e-4ceb-ba09-6a4ab7c0fed5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:07.601828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02c901-9a4e-4ceb-ba09-6a4ab7c0fed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:07.637931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ee02c901-9a4e-4ceb-ba09-6a4ab7c0fed5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:07.640774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ee02c901-9a4e-4ceb-ba09-6a4ab7c0fed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:07.673309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb461952-b504-4421-b9f3-d35682169e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:07.674068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb461952-b504-4421-b9f3-d35682169e12 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:07.674455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb461952-b504-4421-b9f3-d35682169e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:07.674933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb461952-b504-4421-b9f3-d35682169e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:07.676122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb461952-b504-4421-b9f3-d35682169e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:07.679935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb461952-b504-4421-b9f3-d35682169e12 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:07.714466 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:07.743526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 960/1912] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:07.743526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0cc75-4cf8-4c2b-8d3f-725ed9e8e8c4 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:07.743526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0cc75-4cf8-4c2b-8d3f-725ed9e8e8c4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:07.743526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0cc75-4cf8-4c2b-8d3f-725ed9e8e8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:07.778863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-cad0cc75-4cf8-4c2b-8d3f-725ed9e8e8c4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:07.781171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-cad0cc75-4cf8-4c2b-8d3f-725ed9e8e8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:07.795418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 961/1913] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:07.797748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cf4d8-fa5e-4782-b8ae-c43fd04397c8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:07.798242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cf4d8-fa5e-4782-b8ae-c43fd04397c8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:07.798720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8cf4d8-fa5e-4782-b8ae-c43fd04397c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:07.834012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-7c8cf4d8-fa5e-4782-b8ae-c43fd04397c8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:07.836266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7c8cf4d8-fa5e-4782-b8ae-c43fd04397c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:07.845179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 953/1914] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:07 2022] POST /identity/v3/projects => generated 381 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:07.845179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccb351-8e1a-49f6-884e-134508f451e3 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:07.845179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccb351-8e1a-49f6-884e-134508f451e3 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:07.845179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccb351-8e1a-49f6-884e-134508f451e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:07.870065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d5ccb351-8e1a-49f6-884e-134508f451e3 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:07.872405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d5ccb351-8e1a-49f6-884e-134508f451e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:07.956741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 962/1915] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:07.960439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cf4d8-fa5e-4782-b8ae-c43fd04397c8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:07.962463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7c8cf4d8-fa5e-4782-b8ae-c43fd04397c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:07.976150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8cf4d8-fa5e-4782-b8ae-c43fd04397c8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:07.980405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 954/1916] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:07.981502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90248d9b-7a86-4752-8278-aa328ec15d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:07.982195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90248d9b-7a86-4752-8278-aa328ec15d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:07.982320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccb351-8e1a-49f6-884e-134508f451e3 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:07.982470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90248d9b-7a86-4752-8278-aa328ec15d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:07.983273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90248d9b-7a86-4752-8278-aa328ec15d80 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:07.983954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90248d9b-7a86-4752-8278-aa328ec15d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:07.986110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90248d9b-7a86-4752-8278-aa328ec15d80 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.000708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccb351-8e1a-49f6-884e-134508f451e3 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.006764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d948a6b-0aab-4795-a643-c3dbecb12ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.007393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d948a6b-0aab-4795-a643-c3dbecb12ddc admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.007712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d948a6b-0aab-4795-a643-c3dbecb12ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.007984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d948a6b-0aab-4795-a643-c3dbecb12ddc admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.008646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d948a6b-0aab-4795-a643-c3dbecb12ddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:08.008948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d948a6b-0aab-4795-a643-c3dbecb12ddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.010589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d948a6b-0aab-4795-a643-c3dbecb12ddc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.024480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 955/1917] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:08.025383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d948a6b-0aab-4795-a643-c3dbecb12ddc admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.035512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d948a6b-0aab-4795-a643-c3dbecb12ddc admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.040566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14124f59-7bd7-4e62-aa71-47c7e785f818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.041190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14124f59-7bd7-4e62-aa71-47c7e785f818 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.041459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14124f59-7bd7-4e62-aa71-47c7e785f818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.041729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14124f59-7bd7-4e62-aa71-47c7e785f818 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.042391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14124f59-7bd7-4e62-aa71-47c7e785f818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:08.042686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14124f59-7bd7-4e62-aa71-47c7e785f818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.044387 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14124f59-7bd7-4e62-aa71-47c7e785f818 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.060151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 956/1918] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:08.064544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14124f59-7bd7-4e62-aa71-47c7e785f818 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.081192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14124f59-7bd7-4e62-aa71-47c7e785f818 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.086978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f77be-6fe8-4be5-8793-a0f634217923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.087623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f77be-6fe8-4be5-8793-a0f634217923 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.087907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f77be-6fe8-4be5-8793-a0f634217923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.088177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f77be-6fe8-4be5-8793-a0f634217923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.088950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f77be-6fe8-4be5-8793-a0f634217923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.091383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9f77be-6fe8-4be5-8793-a0f634217923 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.104601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 963/1919] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:07 2022] POST /identity/v3/users => generated 297 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:08.104839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90248d9b-7a86-4752-8278-aa328ec15d80 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.116049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90248d9b-7a86-4752-8278-aa328ec15d80 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.122720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad72bda-c696-4f2c-a4c0-f7fc4bdfb034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.124003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad72bda-c696-4f2c-a4c0-f7fc4bdfb034 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.124288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad72bda-c696-4f2c-a4c0-f7fc4bdfb034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.124570 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad72bda-c696-4f2c-a4c0-f7fc4bdfb034 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.125224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad72bda-c696-4f2c-a4c0-f7fc4bdfb034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.127220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad72bda-c696-4f2c-a4c0-f7fc4bdfb034 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.253409 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 964/1920] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:08 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:08.253409 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad72bda-c696-4f2c-a4c0-f7fc4bdfb034 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.255695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 957/1921] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:08 2022] POST /identity/v3/projects => generated 401 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:08.256716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f77be-6fe8-4be5-8793-a0f634217923 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.368177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f77be-6fe8-4be5-8793-a0f634217923 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.386198 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71985ca3-a247-4fec-a69b-2af4fe8cecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.387103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71985ca3-a247-4fec-a69b-2af4fe8cecd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.387584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71985ca3-a247-4fec-a69b-2af4fe8cecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.388039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71985ca3-a247-4fec-a69b-2af4fe8cecd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.388843 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71985ca3-a247-4fec-a69b-2af4fe8cecd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.391624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71985ca3-a247-4fec-a69b-2af4fe8cecd4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.409926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad72bda-c696-4f2c-a4c0-f7fc4bdfb034 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.418848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44296d9-4068-45d9-acb1-7bf397427a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.420149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44296d9-4068-45d9-acb1-7bf397427a75 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.421399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44296d9-4068-45d9-acb1-7bf397427a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.421976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44296d9-4068-45d9-acb1-7bf397427a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.423253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44296d9-4068-45d9-acb1-7bf397427a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.426300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44296d9-4068-45d9-acb1-7bf397427a75 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.451211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 965/1922] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:08.452326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44296d9-4068-45d9-acb1-7bf397427a75 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.471042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44296d9-4068-45d9-acb1-7bf397427a75 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.479114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b33715-aed5-4489-91d2-7b2ec3254dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.480150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b33715-aed5-4489-91d2-7b2ec3254dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.480562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b33715-aed5-4489-91d2-7b2ec3254dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.481023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b33715-aed5-4489-91d2-7b2ec3254dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.482085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b33715-aed5-4489-91d2-7b2ec3254dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.486407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b33715-aed5-4489-91d2-7b2ec3254dc4 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.536865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 958/1923] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:08 2022] POST /identity/v3/users => generated 292 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:08.537867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71985ca3-a247-4fec-a69b-2af4fe8cecd4 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.548869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71985ca3-a247-4fec-a69b-2af4fe8cecd4 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.553136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:08.554434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f319c9-6124-41f6-8daa-215150a7372d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.555184 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f319c9-6124-41f6-8daa-215150a7372d admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.555478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f319c9-6124-41f6-8daa-215150a7372d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.555864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f319c9-6124-41f6-8daa-215150a7372d admin admin] PATH_INFO: `/v3/projects/f63e0549209b45999dd95fe8cfaa80d4/users/75c36d6ce89d41499ff6081eb32f7de4/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.580083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f319c9-6124-41f6-8daa-215150a7372d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63e0549209b45999dd95fe8cfaa80d4, user_id=75c36d6ce89d41499ff6081eb32f7de4, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.582928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f319c9-6124-41f6-8daa-215150a7372d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.638718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 966/1924] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:08 2022] POST /identity/v3/users => generated 301 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:08.640882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b33715-aed5-4489-91d2-7b2ec3254dc4 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.652171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b33715-aed5-4489-91d2-7b2ec3254dc4 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.658213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328bb72-2b1a-4f71-b6bc-151bb6313b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.658451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328bb72-2b1a-4f71-b6bc-151bb6313b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.658609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328bb72-2b1a-4f71-b6bc-151bb6313b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.658838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328bb72-2b1a-4f71-b6bc-151bb6313b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.659501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328bb72-2b1a-4f71-b6bc-151bb6313b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.661260 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328bb72-2b1a-4f71-b6bc-151bb6313b6b admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.672909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 959/1925] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:08 2022] PUT /identity/v3/projects/f63e0549209b45999dd95fe8cfaa80d4/users/75c36d6ce89d41499ff6081eb32f7de4/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:08.672909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f319c9-6124-41f6-8daa-215150a7372d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.677587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 967/1926] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:08.678433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328bb72-2b1a-4f71-b6bc-151bb6313b6b admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.696115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328bb72-2b1a-4f71-b6bc-151bb6313b6b admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.764836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f319c9-6124-41f6-8daa-215150a7372d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.770846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493850e-2895-4f79-b96b-fc88f4e09a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.772078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493850e-2895-4f79-b96b-fc88f4e09a4f admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.772078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493850e-2895-4f79-b96b-fc88f4e09a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.772395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493850e-2895-4f79-b96b-fc88f4e09a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.773154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493850e-2895-4f79-b96b-fc88f4e09a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.775168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493850e-2895-4f79-b96b-fc88f4e09a4f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.796924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 960/1927] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:08.799009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493850e-2895-4f79-b96b-fc88f4e09a4f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.800879 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d5f57-8311-4c23-ace2-2159e0cedf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.801952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d5f57-8311-4c23-ace2-2159e0cedf09 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.802812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d5f57-8311-4c23-ace2-2159e0cedf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.802986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d5f57-8311-4c23-ace2-2159e0cedf09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.805342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d5f57-8311-4c23-ace2-2159e0cedf09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.806743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d5f57-8311-4c23-ace2-2159e0cedf09 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.817266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493850e-2895-4f79-b96b-fc88f4e09a4f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.827418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea32d2-d63a-49b5-8752-39f52a36538c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.828530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea32d2-d63a-49b5-8752-39f52a36538c admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.829037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea32d2-d63a-49b5-8752-39f52a36538c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.829514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea32d2-d63a-49b5-8752-39f52a36538c admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.830737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea32d2-d63a-49b5-8752-39f52a36538c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.832901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 968/1928] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:08.835949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d5f57-8311-4c23-ace2-2159e0cedf09 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.840283 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea32d2-d63a-49b5-8752-39f52a36538c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.856848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d5f57-8311-4c23-ace2-2159e0cedf09 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.860941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 961/1929] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:08.863637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea32d2-d63a-49b5-8752-39f52a36538c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.866970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705690e-4c32-478f-b665-0b827353f435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.875469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705690e-4c32-478f-b665-0b827353f435 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.875469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705690e-4c32-478f-b665-0b827353f435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.875469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2705690e-4c32-478f-b665-0b827353f435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.875469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705690e-4c32-478f-b665-0b827353f435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.875469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2705690e-4c32-478f-b665-0b827353f435 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.883738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea32d2-d63a-49b5-8752-39f52a36538c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.894136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8447ca6-932f-419f-8676-9ab310fb096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.897433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 969/1930] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:08.897897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447ca6-932f-419f-8676-9ab310fb096c admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.898525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447ca6-932f-419f-8676-9ab310fb096c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.899043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8447ca6-932f-419f-8676-9ab310fb096c admin admin] PATH_INFO: `/v3/projects/f63e0549209b45999dd95fe8cfaa80d4/users/75c36d6ce89d41499ff6081eb32f7de4/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.903827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705690e-4c32-478f-b665-0b827353f435 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.904253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8447ca6-932f-419f-8676-9ab310fb096c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63e0549209b45999dd95fe8cfaa80d4, user_id=75c36d6ce89d41499ff6081eb32f7de4, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.909613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8447ca6-932f-419f-8676-9ab310fb096c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:08.919301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2705690e-4c32-478f-b665-0b827353f435 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:08.930255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac737b-603d-49c8-bc8f-0848558971bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:08.931018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac737b-603d-49c8-bc8f-0848558971bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:08.931366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac737b-603d-49c8-bc8f-0848558971bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:08.931740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac737b-603d-49c8-bc8f-0848558971bd admin admin] PATH_INFO: `/v3/projects/6db6561fbd664d1b8d140517991c0f4e/users/eaee6baf818a4c4481637cbfcc16d4fd/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:08.937030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 962/1931] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:08 2022] PUT /identity/v3/projects/f63e0549209b45999dd95fe8cfaa80d4/users/75c36d6ce89d41499ff6081eb32f7de4/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:08.939341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8447ca6-932f-419f-8676-9ab310fb096c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:08.968367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac737b-603d-49c8-bc8f-0848558971bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db6561fbd664d1b8d140517991c0f4e, user_id=eaee6baf818a4c4481637cbfcc16d4fd, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:08.971898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac737b-603d-49c8-bc8f-0848558971bd admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:09.035063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8447ca6-932f-419f-8676-9ab310fb096c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:09.040990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebf567-f557-4f22-bbcb-267469a843bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:09.041937 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebf567-f557-4f22-bbcb-267469a843bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:09.042463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebf567-f557-4f22-bbcb-267469a843bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:09.042976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebf567-f557-4f22-bbcb-267469a843bf admin admin] PATH_INFO: `/v3/projects/bce7263e4aee46beab417b720fefa59d/users/104d81321ef34798b217f034bd4b304e/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:09.068016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebf567-f557-4f22-bbcb-267469a843bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce7263e4aee46beab417b720fefa59d, user_id=104d81321ef34798b217f034bd4b304e, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:09.071359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebf567-f557-4f22-bbcb-267469a843bf admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:09.126348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 970/1932] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:08 2022] PUT /identity/v3/projects/6db6561fbd664d1b8d140517991c0f4e/users/eaee6baf818a4c4481637cbfcc16d4fd/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:09.135857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49803bf-490e-41ec-b510-a9a273ba7be2 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:09.135857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49803bf-490e-41ec-b510-a9a273ba7be2 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:09.135857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49803bf-490e-41ec-b510-a9a273ba7be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:09.164525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 963/1933] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:08 2022] PUT /identity/v3/projects/bce7263e4aee46beab417b720fefa59d/users/104d81321ef34798b217f034bd4b304e/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:09.164525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4666b-c002-4b71-ad2a-d1909f3bc936 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:09.164525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4666b-c002-4b71-ad2a-d1909f3bc936 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:09.164525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4666b-c002-4b71-ad2a-d1909f3bc936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:09.168420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c49803bf-490e-41ec-b510-a9a273ba7be2 None None] MFA Rules not processed for user `75c36d6ce89d41499ff6081eb32f7de4`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:09.171392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c49803bf-490e-41ec-b510-a9a273ba7be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:09.207387 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f6d4666b-c002-4b71-ad2a-d1909f3bc936 None None] MFA Rules not processed for user `eaee6baf818a4c4481637cbfcc16d4fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:09.208932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f6d4666b-c002-4b71-ad2a-d1909f3bc936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:09.297562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 971/1934] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:09.299361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd064f-3c86-4f1d-bcf7-d957b23fbd05 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:09.299907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd064f-3c86-4f1d-bcf7-d957b23fbd05 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:09.300440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afd064f-3c86-4f1d-bcf7-d957b23fbd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:09.322797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 964/1935] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:09.323839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4666b-c002-4b71-ad2a-d1909f3bc936 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:09.333212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f6d4666b-c002-4b71-ad2a-d1909f3bc936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:09.359575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3afd064f-3c86-4f1d-bcf7-d957b23fbd05 None None] MFA Rules not processed for user `104d81321ef34798b217f034bd4b304e`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:09.361096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3afd064f-3c86-4f1d-bcf7-d957b23fbd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:09.422396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4666b-c002-4b71-ad2a-d1909f3bc936 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:09.426631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da79338-bf6b-4372-b215-8e3ed2640fad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:09.427369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da79338-bf6b-4372-b215-8e3ed2640fad service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:09.427756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da79338-bf6b-4372-b215-8e3ed2640fad service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:09.428134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da79338-bf6b-4372-b215-8e3ed2640fad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:09.438674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da79338-bf6b-4372-b215-8e3ed2640fad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:09.440217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da79338-bf6b-4372-b215-8e3ed2640fad service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:09.452604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 965/1936] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:09.454858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da79338-bf6b-4372-b215-8e3ed2640fad service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:09.462933 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 972/1937] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:09 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:09.470653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd064f-3c86-4f1d-bcf7-d957b23fbd05 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:09.537658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:09.605259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da79338-bf6b-4372-b215-8e3ed2640fad service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:09.625159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071106-e254-41d5-8167-677b5f694fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:09.625829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071106-e254-41d5-8167-677b5f694fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:09.626114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071106-e254-41d5-8167-677b5f694fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:09.626393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071106-e254-41d5-8167-677b5f694fe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:09.627969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d071106-e254-41d5-8167-677b5f694fe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:09.638199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d071106-e254-41d5-8167-677b5f694fe9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:09.681667 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afd064f-3c86-4f1d-bcf7-d957b23fbd05 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:09.686889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d98b-5300-4669-9f37-92e75eb7963c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:09.688192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d98b-5300-4669-9f37-92e75eb7963c admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:09.688562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d98b-5300-4669-9f37-92e75eb7963c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:09.689024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d98b-5300-4669-9f37-92e75eb7963c admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:09.690266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6d98b-5300-4669-9f37-92e75eb7963c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:09.694385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6d98b-5300-4669-9f37-92e75eb7963c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:09.822634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 966/1938] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:09 2022] POST /identity/v3/projects => generated 385 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:09.831651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071106-e254-41d5-8167-677b5f694fe9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:09.874332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 973/1939] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:09 2022] POST /identity/v3/projects => generated 401 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:31:09.874968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d98b-5300-4669-9f37-92e75eb7963c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:09.975358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071106-e254-41d5-8167-677b5f694fe9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:09.980245 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d98b-5300-4669-9f37-92e75eb7963c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:09.983960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe23c-a88c-4d28-852e-c172dc15b809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:09.985005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe23c-a88c-4d28-852e-c172dc15b809 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:09.985550 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe23c-a88c-4d28-852e-c172dc15b809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:09.985668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1760684-d8da-48ad-a6b2-2ecdc92422ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:09.986082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe23c-a88c-4d28-852e-c172dc15b809 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:09.986346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1760684-d8da-48ad-a6b2-2ecdc92422ad admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:09.986640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1760684-d8da-48ad-a6b2-2ecdc92422ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:09.986904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1760684-d8da-48ad-a6b2-2ecdc92422ad admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:09.987272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbe23c-a88c-4d28-852e-c172dc15b809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:09.987544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1760684-d8da-48ad-a6b2-2ecdc92422ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:09.989541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1760684-d8da-48ad-a6b2-2ecdc92422ad admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:09.991291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbe23c-a88c-4d28-852e-c172dc15b809 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:10.022397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:10.124540 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 974/1940] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:09 2022] POST /identity/v3/users => generated 307 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:10.128533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 967/1941] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:09 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:10.130219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe23c-a88c-4d28-852e-c172dc15b809 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:10.131069 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1760684-d8da-48ad-a6b2-2ecdc92422ad admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:10.143519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe23c-a88c-4d28-852e-c172dc15b809 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:10.149691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f205f19-98b0-4e77-a848-28ebfc5fe2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:10.149997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1760684-d8da-48ad-a6b2-2ecdc92422ad admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:10.150816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f205f19-98b0-4e77-a848-28ebfc5fe2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:10.151325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f205f19-98b0-4e77-a848-28ebfc5fe2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:10.151832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f205f19-98b0-4e77-a848-28ebfc5fe2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:10.153079 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f205f19-98b0-4e77-a848-28ebfc5fe2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:10.155595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f205f19-98b0-4e77-a848-28ebfc5fe2f9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:10.156545 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cdf44-77dc-4a74-996f-22d0ba8a3ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:10.159015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cdf44-77dc-4a74-996f-22d0ba8a3ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:10.159015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cdf44-77dc-4a74-996f-22d0ba8a3ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:10.159015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63cdf44-77dc-4a74-996f-22d0ba8a3ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:10.159015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cdf44-77dc-4a74-996f-22d0ba8a3ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:10.160735 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63cdf44-77dc-4a74-996f-22d0ba8a3ac1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:10.175601 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 968/1942] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:10.175830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 975/1943] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:10.180318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f205f19-98b0-4e77-a848-28ebfc5fe2f9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:10.181378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cdf44-77dc-4a74-996f-22d0ba8a3ac1 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:10.198018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f205f19-98b0-4e77-a848-28ebfc5fe2f9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:10.200564 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63cdf44-77dc-4a74-996f-22d0ba8a3ac1 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:10.206656 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17e027-4fd3-49a6-9052-e1721461119c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:10.207925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17e027-4fd3-49a6-9052-e1721461119c admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:10.208124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17e027-4fd3-49a6-9052-e1721461119c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:10.208975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17e027-4fd3-49a6-9052-e1721461119c admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:10.209712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17e027-4fd3-49a6-9052-e1721461119c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:10.209844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b81607-74d7-423d-b6df-b0fc041747c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:10.211006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b81607-74d7-423d-b6df-b0fc041747c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:10.220206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b81607-74d7-423d-b6df-b0fc041747c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:10.220551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17e027-4fd3-49a6-9052-e1721461119c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:10.220644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b81607-74d7-423d-b6df-b0fc041747c6 admin admin] PATH_INFO: `/v3/projects/3c7a8dbc4e7a42f0822dfdc780fa1172/users/a85961ae47944da09f5c2bcc80cb271a/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:10.234199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 969/1944] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:10.235454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17e027-4fd3-49a6-9052-e1721461119c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:10.250095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b81607-74d7-423d-b6df-b0fc041747c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7a8dbc4e7a42f0822dfdc780fa1172, user_id=a85961ae47944da09f5c2bcc80cb271a, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:10.253114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17e027-4fd3-49a6-9052-e1721461119c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:10.255855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b81607-74d7-423d-b6df-b0fc041747c6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:10.258764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f081166-ca4c-4f6a-a129-701b2be9b825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:10.261170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f081166-ca4c-4f6a-a129-701b2be9b825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:10.261170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f081166-ca4c-4f6a-a129-701b2be9b825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:10.261170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f081166-ca4c-4f6a-a129-701b2be9b825 admin admin] PATH_INFO: `/v3/projects/37b23fdc79934b9cb25959e8407dfcef/users/7324b8c749334164890fcb6c900122ce/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:10.281578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f081166-ca4c-4f6a-a129-701b2be9b825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37b23fdc79934b9cb25959e8407dfcef, user_id=7324b8c749334164890fcb6c900122ce, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:10.284986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f081166-ca4c-4f6a-a129-701b2be9b825 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:10.384190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:10.392741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 970/1945] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:10 2022] PUT /identity/v3/projects/37b23fdc79934b9cb25959e8407dfcef/users/7324b8c749334164890fcb6c900122ce/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:10.397595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 976/1946] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:10 2022] PUT /identity/v3/projects/3c7a8dbc4e7a42f0822dfdc780fa1172/users/a85961ae47944da09f5c2bcc80cb271a/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:10.402305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f081166-ca4c-4f6a-a129-701b2be9b825 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:10.411052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79357029-d749-4ac2-90df-aaf2e02c9d41 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:10.411749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79357029-d749-4ac2-90df-aaf2e02c9d41 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:10.412115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79357029-d749-4ac2-90df-aaf2e02c9d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:10.458236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-79357029-d749-4ac2-90df-aaf2e02c9d41 None None] MFA Rules not processed for user `7324b8c749334164890fcb6c900122ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:10.460429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-79357029-d749-4ac2-90df-aaf2e02c9d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:10.499579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f081166-ca4c-4f6a-a129-701b2be9b825 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:10.506261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87453709-0385-4a33-b054-39ea32d4e683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:10.507249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87453709-0385-4a33-b054-39ea32d4e683 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:10.507694 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87453709-0385-4a33-b054-39ea32d4e683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:10.508135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87453709-0385-4a33-b054-39ea32d4e683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:10.509155 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87453709-0385-4a33-b054-39ea32d4e683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:10.510975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87453709-0385-4a33-b054-39ea32d4e683 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:10.531577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 971/1947] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:10.536692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87453709-0385-4a33-b054-39ea32d4e683 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:10.553236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87453709-0385-4a33-b054-39ea32d4e683 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:10.560329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9a181-562a-4061-ba46-1cfd8bcb3d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:10.561155 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9a181-562a-4061-ba46-1cfd8bcb3d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:10.561775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9a181-562a-4061-ba46-1cfd8bcb3d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:10.562112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9a181-562a-4061-ba46-1cfd8bcb3d8f admin admin] PATH_INFO: `/v3/projects/3c7a8dbc4e7a42f0822dfdc780fa1172/users/a85961ae47944da09f5c2bcc80cb271a/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:10.564953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9a181-562a-4061-ba46-1cfd8bcb3d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7a8dbc4e7a42f0822dfdc780fa1172, user_id=a85961ae47944da09f5c2bcc80cb271a, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:10.565236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 977/1948] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:10.568964 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf9a181-562a-4061-ba46-1cfd8bcb3d8f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:10.570260 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79357029-d749-4ac2-90df-aaf2e02c9d41 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:10.584035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79357029-d749-4ac2-90df-aaf2e02c9d41 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:10.586081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 972/1949] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:10 2022] PUT /identity/v3/projects/3c7a8dbc4e7a42f0822dfdc780fa1172/users/a85961ae47944da09f5c2bcc80cb271a/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:10.604965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d6558-a815-407d-a8f3-954fa68af2c1 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:10.605896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d6558-a815-407d-a8f3-954fa68af2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:10.606505 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d6558-a815-407d-a8f3-954fa68af2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:10.662183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a06d6558-a815-407d-a8f3-954fa68af2c1 None None] MFA Rules not processed for user `a85961ae47944da09f5c2bcc80cb271a`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:10.663766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a06d6558-a815-407d-a8f3-954fa68af2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:10.709956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95e42e-50cf-4e7c-81a9-6d014b6ba653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:10.710842 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95e42e-50cf-4e7c-81a9-6d014b6ba653 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:10.711207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95e42e-50cf-4e7c-81a9-6d014b6ba653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:10.711588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95e42e-50cf-4e7c-81a9-6d014b6ba653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:10.712588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95e42e-50cf-4e7c-81a9-6d014b6ba653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:10.714658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95e42e-50cf-4e7c-81a9-6d014b6ba653 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:10.850362 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 973/1950] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:10 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:10.893221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:10.930932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 978/1951] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:10 2022] POST /identity/v3/projects => generated 385 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:10.932537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d6558-a815-407d-a8f3-954fa68af2c1 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:10.948292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d6558-a815-407d-a8f3-954fa68af2c1 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:11.025645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995584d4-e56e-48a4-b858-3895389f9da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:11.026346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995584d4-e56e-48a4-b858-3895389f9da3 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:11.026530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995584d4-e56e-48a4-b858-3895389f9da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:11.026808 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995584d4-e56e-48a4-b858-3895389f9da3 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:11.027449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995584d4-e56e-48a4-b858-3895389f9da3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:11.029430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995584d4-e56e-48a4-b858-3895389f9da3 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:11.090452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:11.144057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95e42e-50cf-4e7c-81a9-6d014b6ba653 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:11.147894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-db95e42e-50cf-4e7c-81a9-6d014b6ba653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:11.184585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 974/1952] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:10 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:11.189401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995584d4-e56e-48a4-b858-3895389f9da3 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:11.200546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995584d4-e56e-48a4-b858-3895389f9da3 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:11.215013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00671716-19b5-446d-8f42-da321924e10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:11.216886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00671716-19b5-446d-8f42-da321924e10c admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:11.216886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00671716-19b5-446d-8f42-da321924e10c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:11.216886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00671716-19b5-446d-8f42-da321924e10c admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:11.218428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00671716-19b5-446d-8f42-da321924e10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:11.220292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00671716-19b5-446d-8f42-da321924e10c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:11.223752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95e42e-50cf-4e7c-81a9-6d014b6ba653 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:11.230587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6897fc-66c8-4423-86b2-671ea3429a5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:11.231748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6897fc-66c8-4423-86b2-671ea3429a5b service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:11.232262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6897fc-66c8-4423-86b2-671ea3429a5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:11.232781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6897fc-66c8-4423-86b2-671ea3429a5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:11.256383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 975/1953] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:11.257075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00671716-19b5-446d-8f42-da321924e10c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:11.282063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00671716-19b5-446d-8f42-da321924e10c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:11.289431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d34153-b00a-477e-8e62-c7e7d11ecd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:11.290236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d34153-b00a-477e-8e62-c7e7d11ecd89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:11.290616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d34153-b00a-477e-8e62-c7e7d11ecd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:11.290967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d34153-b00a-477e-8e62-c7e7d11ecd89 admin admin] PATH_INFO: `/v3/projects/4796b1219050424888e9e474a7df1b66/users/6c3ae649482548b185322ba02f298521/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:11.313426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d34153-b00a-477e-8e62-c7e7d11ecd89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4796b1219050424888e9e474a7df1b66, user_id=6c3ae649482548b185322ba02f298521, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:11.317763 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d34153-b00a-477e-8e62-c7e7d11ecd89 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:11.379793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6897fc-66c8-4423-86b2-671ea3429a5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:11.382335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6897fc-66c8-4423-86b2-671ea3429a5b service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:11.395921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 979/1954] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:11 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:11.396845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6897fc-66c8-4423-86b2-671ea3429a5b service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:11.480030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 976/1955] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:11 2022] PUT /identity/v3/projects/4796b1219050424888e9e474a7df1b66/users/6c3ae649482548b185322ba02f298521/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:11.481543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6897fc-66c8-4423-86b2-671ea3429a5b service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:11.485416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d34153-b00a-477e-8e62-c7e7d11ecd89 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:11.595782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300fd68-8d9b-4d9c-b36b-d4ef816d7dea tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c36d6ce89d41499ff6081eb32f7de4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f63e0549209b45999dd95fe8cfaa80d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:11.597874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300fd68-8d9b-4d9c-b36b-d4ef816d7dea tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:11.597874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300fd68-8d9b-4d9c-b36b-d4ef816d7dea tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:11.597874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300fd68-8d9b-4d9c-b36b-d4ef816d7dea tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:11.599867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300fd68-8d9b-4d9c-b36b-d4ef816d7dea tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:11.600449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300fd68-8d9b-4d9c-b36b-d4ef816d7dea tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:11.603783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300fd68-8d9b-4d9c-b36b-d4ef816d7dea tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:11.604702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d34153-b00a-477e-8e62-c7e7d11ecd89 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:11.613375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f875d-333f-4dd0-897e-e841b3c495cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:11.614395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f875d-333f-4dd0-897e-e841b3c495cb admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:11.614913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f875d-333f-4dd0-897e-e841b3c495cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:11.615393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f875d-333f-4dd0-897e-e841b3c495cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:11.616774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f875d-333f-4dd0-897e-e841b3c495cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:11.618783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f875d-333f-4dd0-897e-e841b3c495cb admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:11.635615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 980/1956] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:11.637640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 977/1957] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:11.641657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300fd68-8d9b-4d9c-b36b-d4ef816d7dea tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:11.643882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f875d-333f-4dd0-897e-e841b3c495cb admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:11.659153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300fd68-8d9b-4d9c-b36b-d4ef816d7dea tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:11.662313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f875d-333f-4dd0-897e-e841b3c495cb admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:11.669608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fa78b-31b1-4c09-859d-5d77354c7753 tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c36d6ce89d41499ff6081eb32f7de4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f63e0549209b45999dd95fe8cfaa80d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:11.671114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa78b-31b1-4c09-859d-5d77354c7753 tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:11.671615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa78b-31b1-4c09-859d-5d77354c7753 tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:11.672084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a6281-4806-4dcd-9e44-35aad27a1759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:11.673272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa78b-31b1-4c09-859d-5d77354c7753 tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:11.673725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fa78b-31b1-4c09-859d-5d77354c7753 tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:11.673832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6281-4806-4dcd-9e44-35aad27a1759 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:11.674432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6281-4806-4dcd-9e44-35aad27a1759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:11.674977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a6281-4806-4dcd-9e44-35aad27a1759 admin admin] PATH_INFO: `/v3/projects/4796b1219050424888e9e474a7df1b66/users/6c3ae649482548b185322ba02f298521/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:11.677700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fa78b-31b1-4c09-859d-5d77354c7753 tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:11.678912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a6281-4806-4dcd-9e44-35aad27a1759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4796b1219050424888e9e474a7df1b66, user_id=6c3ae649482548b185322ba02f298521, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:11.683356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a6281-4806-4dcd-9e44-35aad27a1759 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:11.713872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 978/1958] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:11 2022] PUT /identity/v3/projects/4796b1219050424888e9e474a7df1b66/users/6c3ae649482548b185322ba02f298521/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:11.715624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509f3e8-9042-4bbb-ac0d-07083f523684 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:11.716237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509f3e8-9042-4bbb-ac0d-07083f523684 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:11.716539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509f3e8-9042-4bbb-ac0d-07083f523684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:11.727320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:11.763517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-9509f3e8-9042-4bbb-ac0d-07083f523684 None None] MFA Rules not processed for user `6c3ae649482548b185322ba02f298521`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:11.765681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9509f3e8-9042-4bbb-ac0d-07083f523684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:11.843235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 981/1959] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:11 2022] POST /identity/v3/projects => generated 354 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:31:11.847172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fa78b-31b1-4c09-859d-5d77354c7753 tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:11.891346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 979/1960] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:11.891346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509f3e8-9042-4bbb-ac0d-07083f523684 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:11.945293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fa78b-31b1-4c09-859d-5d77354c7753 tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:11.949883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a595a3-cfd8-4e0e-8f22-df6cb26ca4eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:11.950716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a595a3-cfd8-4e0e-8f22-df6cb26ca4eb service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:11.952072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a595a3-cfd8-4e0e-8f22-df6cb26ca4eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:11.952072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a595a3-cfd8-4e0e-8f22-df6cb26ca4eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:11.975277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:12.010371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:12.015038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509f3e8-9042-4bbb-ac0d-07083f523684 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:12.042186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a595a3-cfd8-4e0e-8f22-df6cb26ca4eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:12.044081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a595a3-cfd8-4e0e-8f22-df6cb26ca4eb service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:12.056169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 982/1961] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:11 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:12.057023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a595a3-cfd8-4e0e-8f22-df6cb26ca4eb service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:12.067964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a595a3-cfd8-4e0e-8f22-df6cb26ca4eb service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:12.074984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c11f5a-1f26-4ba4-9d72-04943443ba95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:12.076163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c11f5a-1f26-4ba4-9d72-04943443ba95 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:12.077660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c11f5a-1f26-4ba4-9d72-04943443ba95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:12.078120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c11f5a-1f26-4ba4-9d72-04943443ba95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:12.115281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452d012-5133-4c60-bbb0-6b20482c273c tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c36d6ce89d41499ff6081eb32f7de4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f63e0549209b45999dd95fe8cfaa80d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:12.116210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d012-5133-4c60-bbb0-6b20482c273c tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:12.116556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d012-5133-4c60-bbb0-6b20482c273c tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:12.116897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452d012-5133-4c60-bbb0-6b20482c273c tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:12.117718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452d012-5133-4c60-bbb0-6b20482c273c tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:12.118207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452d012-5133-4c60-bbb0-6b20482c273c tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:12.120066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452d012-5133-4c60-bbb0-6b20482c273c tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:12.141806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 980/1962] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:12.185381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c11f5a-1f26-4ba4-9d72-04943443ba95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:12.187085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c11f5a-1f26-4ba4-9d72-04943443ba95 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:12.199827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 983/1963] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:12.210403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:12.236495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:12.612626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:13.011480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:13.417180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452d012-5133-4c60-bbb0-6b20482c273c tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:13.465368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452d012-5133-4c60-bbb0-6b20482c273c tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:13.473186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:13.486882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75c36d6ce89d41499ff6081eb32f7de4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f63e0549209b45999dd95fe8cfaa80d4', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:13.488861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:13.490659 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:13.494249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] PATH_INFO: `/v3/projects/0d258589616241eea5d497ba232c9fba` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:13.499778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0d258589616241eea5d497ba232c9fba)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:13.509804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:13.699604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:13.730755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d258589616241eea5d497ba232c9fba', 'request_id': 'req-b89e11c1-2a03-4e95-b70f-675ac947cb4d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:13.738882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d258589616241eea5d497ba232c9fba', 'request_id': 'req-b89e11c1-2a03-4e95-b70f-675ac947cb4d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:13.863042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c11f5a-1f26-4ba4-9d72-04943443ba95 service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:13.865539 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d8c11f5a-1f26-4ba4-9d72-04943443ba95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:13.881001 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d258589616241eea5d497ba232c9fba', 'request_id': 'req-b89e11c1-2a03-4e95-b70f-675ac947cb4d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:13.902199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] The token cache is being invalidate because project 0d258589616241eea5d497ba232c9fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:13.902488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:13.903188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b89e11c1-2a03-4e95-b70f-675ac947cb4d tempest-VolumeQuotaClassesTest-1210262882 tempest-VolumeQuotaClassesTest-1210262882-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:13.905218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 981/1964] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:13 2022] DELETE /identity/v3/projects/0d258589616241eea5d497ba232c9fba => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:13.954594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c11f5a-1f26-4ba4-9d72-04943443ba95 service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:14.042207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:14.043251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:14.043819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:14.044325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] PATH_INFO: `/v3/users/f67f7b66a8b745db827c0f4d9db181f6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:14.047155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f67f7b66a8b745db827c0f4d9db181f6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:14.051037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:14.115495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:14.335246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f67f7b66a8b745db827c0f4d9db181f6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:14.336091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f67f7b66a8b745db827c0f4d9db181f6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:14.336792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f67f7b66a8b745db827c0f4d9db181f6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:14.348522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f67f7b66a8b745db827c0f4d9db181f6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:14.368144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f67f7b66a8b745db827c0f4d9db181f6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:14.508268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 984/1965] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:13 2022] DELETE /identity/v3/users/f67f7b66a8b745db827c0f4d9db181f6 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:14.513871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:14.515043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0b628-5e02-49f4-9927-259880bc181e None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:14.515939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0b628-5e02-49f4-9927-259880bc181e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:14.516519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0b628-5e02-49f4-9927-259880bc181e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:14.550568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-3af0b628-5e02-49f4-9927-259880bc181e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:14.552291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3af0b628-5e02-49f4-9927-259880bc181e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:14.642359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:14.657704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 982/1966] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:14.670639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:14.672743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:14.765562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836a66f-7dc8-454b-9107-55b52e5b318e admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:14.773138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e370d2-05fe-4e1e-8c76-c1a3852244fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:14.773942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e370d2-05fe-4e1e-8c76-c1a3852244fe service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:14.774369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e370d2-05fe-4e1e-8c76-c1a3852244fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:14.774737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e370d2-05fe-4e1e-8c76-c1a3852244fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:14.786943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e370d2-05fe-4e1e-8c76-c1a3852244fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:14.788849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e370d2-05fe-4e1e-8c76-c1a3852244fe service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:14.806498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 985/1967] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:14.831965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0b628-5e02-49f4-9927-259880bc181e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:14.837074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3af0b628-5e02-49f4-9927-259880bc181e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:14.877438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0b628-5e02-49f4-9927-259880bc181e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:14.895829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:14.897028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:14.897503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:14.897982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] PATH_INFO: `/v3/users/e41b7b6f256c422299984fdd1fffe7bb` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:14.901293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41b7b6f256c422299984fdd1fffe7bb)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:14.906013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:14.970816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:15.155858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:15.198655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41b7b6f256c422299984fdd1fffe7bb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:15.313853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41b7b6f256c422299984fdd1fffe7bb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:15.321298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41b7b6f256c422299984fdd1fffe7bb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:15.349310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41b7b6f256c422299984fdd1fffe7bb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:15.350769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41b7b6f256c422299984fdd1fffe7bb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:15.376378 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 983/1968] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:14 2022] DELETE /identity/v3/users/e41b7b6f256c422299984fdd1fffe7bb => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:15.383622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3954b-b1f5-4747-b45d-d80a463b744d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:15.384173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3954b-b1f5-4747-b45d-d80a463b744d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:15.384586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3954b-b1f5-4747-b45d-d80a463b744d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:15.412301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-85a3954b-b1f5-4747-b45d-d80a463b744d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:15.413888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-85a3954b-b1f5-4747-b45d-d80a463b744d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:15.538018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:15.564296 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 986/1969] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:15.572152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:15.585463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:15.588285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:15.716424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3954b-b1f5-4747-b45d-d80a463b744d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:15.720486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-85a3954b-b1f5-4747-b45d-d80a463b744d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:15.728627 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc674b16-48b9-418b-bbbf-342a11e6cc67 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:15.735464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144f199-e115-417a-8ea9-c90983510107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:15.736714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144f199-e115-417a-8ea9-c90983510107 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:15.737984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144f199-e115-417a-8ea9-c90983510107 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:15.737984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144f199-e115-417a-8ea9-c90983510107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:15.745163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3954b-b1f5-4747-b45d-d80a463b744d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:15.754788 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144f199-e115-417a-8ea9-c90983510107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:15.755611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:15.756575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:15.756962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144f199-e115-417a-8ea9-c90983510107 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:15.757168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:15.757592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] PATH_INFO: `/v3/projects/c0c46cf4b4964719b2108816d34b30dd` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:15.759031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0c46cf4b4964719b2108816d34b30dd)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:15.761447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:15.776375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 984/1970] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:15.904370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0c46cf4b4964719b2108816d34b30dd', 'request_id': 'req-2da8aecd-2d70-4deb-935e-6ea068a6ad63'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:15.912752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0c46cf4b4964719b2108816d34b30dd', 'request_id': 'req-2da8aecd-2d70-4deb-935e-6ea068a6ad63'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.045135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0c46cf4b4964719b2108816d34b30dd', 'request_id': 'req-2da8aecd-2d70-4deb-935e-6ea068a6ad63'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.048294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:16.062324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] The token cache is being invalidate because project c0c46cf4b4964719b2108816d34b30dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:16.062704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.063433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.065451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 987/1971] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:15 2022] DELETE /identity/v3/projects/c0c46cf4b4964719b2108816d34b30dd => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:16.070338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144f199-e115-417a-8ea9-c90983510107 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:16.091452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144f199-e115-417a-8ea9-c90983510107 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:16.144116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:16.146689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:16.207714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:16.239697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:16.240906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:16.241501 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:16.242214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] PATH_INFO: `/v3/users/d6cb65cb9d914b8a80296e9311ef1212` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:16.245196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6cb65cb9d914b8a80296e9311ef1212)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:16.249969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:16.291188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:16.297037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8aecd-2d70-4deb-935e-6ea068a6ad63 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:16.305873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f97ed-335a-484b-93f1-91520a58cec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:16.308148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f97ed-335a-484b-93f1-91520a58cec5 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:16.309252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f97ed-335a-484b-93f1-91520a58cec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:16.309763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f97ed-335a-484b-93f1-91520a58cec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:16.313490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-df1f97ed-335a-484b-93f1-91520a58cec5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:16.426187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f97ed-335a-484b-93f1-91520a58cec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:16.428726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f97ed-335a-484b-93f1-91520a58cec5 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:16.442564 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 988/1972] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:16.521147 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f97ed-335a-484b-93f1-91520a58cec5 service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:16.522915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-df1f97ed-335a-484b-93f1-91520a58cec5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:16.551692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f97ed-335a-484b-93f1-91520a58cec5 service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:16.561666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:16.562377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:16.563000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:16.563342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] PATH_INFO: `/v3/projects/bf7ce4be9dd040f7bbe58e1c93ed5881` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:16.564652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6cb65cb9d914b8a80296e9311ef1212'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.564965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf7ce4be9dd040f7bbe58e1c93ed5881)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:16.567798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:16.645250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:16.682041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6cb65cb9d914b8a80296e9311ef1212'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.683046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6cb65cb9d914b8a80296e9311ef1212'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.704404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:16.719772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf7ce4be9dd040f7bbe58e1c93ed5881', 'request_id': 'req-2a057a12-96a6-46e6-83b9-c75c767c4a1a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.722643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6cb65cb9d914b8a80296e9311ef1212'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.724504 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6cb65cb9d914b8a80296e9311ef1212'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.734454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf7ce4be9dd040f7bbe58e1c93ed5881', 'request_id': 'req-2a057a12-96a6-46e6-83b9-c75c767c4a1a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.759668 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 985/1973] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:16 2022] DELETE /identity/v3/users/d6cb65cb9d914b8a80296e9311ef1212 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:16.895304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf7ce4be9dd040f7bbe58e1c93ed5881', 'request_id': 'req-2a057a12-96a6-46e6-83b9-c75c767c4a1a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.910685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] The token cache is being invalidate because project bf7ce4be9dd040f7bbe58e1c93ed5881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:16.911812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.911812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:16.913921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 989/1974] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:16 2022] DELETE /identity/v3/projects/bf7ce4be9dd040f7bbe58e1c93ed5881 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:16.918550 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:16.920805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:17.013702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2b245-82f3-4e20-8258-d96e48cdf104 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:17.021310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:17.022366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:17.022707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:17.023076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] PATH_INFO: `/v3/users/75c36d6ce89d41499ff6081eb32f7de4` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:17.024656 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c36d6ce89d41499ff6081eb32f7de4)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:17.026865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:17.383047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c36d6ce89d41499ff6081eb32f7de4'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:17.598722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c36d6ce89d41499ff6081eb32f7de4'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:17.599667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c36d6ce89d41499ff6081eb32f7de4'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:17.620389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c36d6ce89d41499ff6081eb32f7de4'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:17.621602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c36d6ce89d41499ff6081eb32f7de4'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:17.646977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 986/1975] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:16 2022] DELETE /identity/v3/users/75c36d6ce89d41499ff6081eb32f7de4 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:18.135962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:18.144492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:18.286780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a057a12-96a6-46e6-83b9-c75c767c4a1a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:18.360582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:18.362002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:18.362002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:18.362002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] PATH_INFO: `/v3/projects/d0288b9bb60049c2bc2d4732eed44f78` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:18.377050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0288b9bb60049c2bc2d4732eed44f78)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:18.380663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:18.567797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:18.570939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:18.594525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17456d15-f724-41a1-8a0b-e2ac52e5b152 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:18.598864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0288b9bb60049c2bc2d4732eed44f78', 'request_id': 'req-1344b748-e6d1-49f6-a175-a2f23b3d2c46'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:18.607075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0288b9bb60049c2bc2d4732eed44f78', 'request_id': 'req-1344b748-e6d1-49f6-a175-a2f23b3d2c46'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:18.667172 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:18.670957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:18.671311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:18.671629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] PATH_INFO: `/v3/projects/f63e0549209b45999dd95fe8cfaa80d4` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:18.672977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63e0549209b45999dd95fe8cfaa80d4)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:18.682258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:18.770785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0288b9bb60049c2bc2d4732eed44f78', 'request_id': 'req-1344b748-e6d1-49f6-a175-a2f23b3d2c46'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:18.772139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:18.784420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] The token cache is being invalidate because project d0288b9bb60049c2bc2d4732eed44f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:18.784843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:18.785747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1344b748-e6d1-49f6-a175-a2f23b3d2c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:18.787850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 990/1976] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:18 2022] DELETE /identity/v3/projects/d0288b9bb60049c2bc2d4732eed44f78 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:18.793627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8de279-edbd-4f09-a447-7791eb8a2eef None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:18.793995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8de279-edbd-4f09-a447-7791eb8a2eef None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:18.794355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8de279-edbd-4f09-a447-7791eb8a2eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:18.827247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6a8de279-edbd-4f09-a447-7791eb8a2eef None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:18.834682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6a8de279-edbd-4f09-a447-7791eb8a2eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:18.844584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 991/1977] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:18.859030 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dd5c4-ca7f-44b0-9aff-a2c2aa5201c2 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:18.860079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dd5c4-ca7f-44b0-9aff-a2c2aa5201c2 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:18.860605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69dd5c4-ca7f-44b0-9aff-a2c2aa5201c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:18.883196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63e0549209b45999dd95fe8cfaa80d4', 'request_id': 'req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:18.889644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63e0549209b45999dd95fe8cfaa80d4', 'request_id': 'req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:18.900954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f69dd5c4-ca7f-44b0-9aff-a2c2aa5201c2 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:18.903368 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f69dd5c4-ca7f-44b0-9aff-a2c2aa5201c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:18.995702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63e0549209b45999dd95fe8cfaa80d4', 'request_id': 'req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:19.011936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 992/1978] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:19.017637 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dd5c4-ca7f-44b0-9aff-a2c2aa5201c2 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:19.029769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69dd5c4-ca7f-44b0-9aff-a2c2aa5201c2 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:19.042912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] The token cache is being invalidate because project f63e0549209b45999dd95fe8cfaa80d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:19.043528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:19.044316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a3d03e9d-ebac-4fce-8f5e-c277cb55ef8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:19.045927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 987/1979] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:18 2022] DELETE /identity/v3/projects/f63e0549209b45999dd95fe8cfaa80d4 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:19.050937 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d0ea7-2252-4ccd-9c9c-a0dd780ffd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:19.051744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d0ea7-2252-4ccd-9c9c-a0dd780ffd2c admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:19.052015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d0ea7-2252-4ccd-9c9c-a0dd780ffd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:19.052345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d0ea7-2252-4ccd-9c9c-a0dd780ffd2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:19.053140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d0ea7-2252-4ccd-9c9c-a0dd780ffd2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:19.053497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d0ea7-2252-4ccd-9c9c-a0dd780ffd2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:19.056203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d0ea7-2252-4ccd-9c9c-a0dd780ffd2c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:19.057012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791ada5-cc5c-4a3a-b96e-be630d3eebef None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:19.057828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791ada5-cc5c-4a3a-b96e-be630d3eebef None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:19.060206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791ada5-cc5c-4a3a-b96e-be630d3eebef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:19.077437 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 993/1980] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:19.081591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d0ea7-2252-4ccd-9c9c-a0dd780ffd2c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:19.083857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b67d0ea7-2252-4ccd-9c9c-a0dd780ffd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:19.099128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e791ada5-cc5c-4a3a-b96e-be630d3eebef None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:19.101009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e791ada5-cc5c-4a3a-b96e-be630d3eebef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:19.127612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 988/1981] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:19.127612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791ada5-cc5c-4a3a-b96e-be630d3eebef None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:19.127612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e791ada5-cc5c-4a3a-b96e-be630d3eebef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:19.143353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d0ea7-2252-4ccd-9c9c-a0dd780ffd2c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:19.144836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791ada5-cc5c-4a3a-b96e-be630d3eebef None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:19.150464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:19.151395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:19.151838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:19.152129 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] PATH_INFO: `/v3/users/104d81321ef34798b217f034bd4b304e` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:19.152769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976c69ff-47ed-4855-a61d-23850e22783b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:19.153457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=104d81321ef34798b217f034bd4b304e)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:19.153981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c69ff-47ed-4855-a61d-23850e22783b admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:19.155037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c69ff-47ed-4855-a61d-23850e22783b admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:19.155864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976c69ff-47ed-4855-a61d-23850e22783b admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:19.156050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:19.157404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c69ff-47ed-4855-a61d-23850e22783b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:19.178418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976c69ff-47ed-4855-a61d-23850e22783b admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:19.195341 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:19.310822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 994/1982] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:19 2022] POST /identity/v3/projects => generated 379 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:31:19.312538 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cffd36-5ffd-421c-8e19-71f002077c4d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:19.312856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cffd36-5ffd-421c-8e19-71f002077c4d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:19.313133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cffd36-5ffd-421c-8e19-71f002077c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:19.335527 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-51cffd36-5ffd-421c-8e19-71f002077c4d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:19.336851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-51cffd36-5ffd-421c-8e19-71f002077c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:19.410934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 995/1983] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:19.411925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cffd36-5ffd-421c-8e19-71f002077c4d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:19.413329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-51cffd36-5ffd-421c-8e19-71f002077c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:19.462013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '104d81321ef34798b217f034bd4b304e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:19.502538 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cffd36-5ffd-421c-8e19-71f002077c4d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:19.506756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd3fdc-5642-40ec-9f43-93813f3a755b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:19.507412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd3fdc-5642-40ec-9f43-93813f3a755b service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:19.507735 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd3fdc-5642-40ec-9f43-93813f3a755b service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:19.508014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd3fdc-5642-40ec-9f43-93813f3a755b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:19.510844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-87fd3fdc-5642-40ec-9f43-93813f3a755b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:19.554196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '104d81321ef34798b217f034bd4b304e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:19.555407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '104d81321ef34798b217f034bd4b304e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:19.587788 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '104d81321ef34798b217f034bd4b304e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:19.590067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '104d81321ef34798b217f034bd4b304e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:19.627384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 989/1984] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:19 2022] DELETE /identity/v3/users/104d81321ef34798b217f034bd4b304e => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:19.631959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:19.634942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:19.651136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd3fdc-5642-40ec-9f43-93813f3a755b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:19.654114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd3fdc-5642-40ec-9f43-93813f3a755b service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:19.683481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 996/1985] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:19.685901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd3fdc-5642-40ec-9f43-93813f3a755b service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:19.722576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd3fdc-5642-40ec-9f43-93813f3a755b service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:19.789510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897596ca-bffe-41cd-98dc-a7b34890b396 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:19.797754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c8a42-d9ad-401b-b269-64058bc4da80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:19.798560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c8a42-d9ad-401b-b269-64058bc4da80 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:19.798917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c8a42-d9ad-401b-b269-64058bc4da80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:19.799396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c8a42-d9ad-401b-b269-64058bc4da80 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:19.800144 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c8a42-d9ad-401b-b269-64058bc4da80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:19.802143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c8a42-d9ad-401b-b269-64058bc4da80 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:19.833139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1b22a-3bd0-409f-80a8-ca3667d2e45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:19.833701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1b22a-3bd0-409f-80a8-ca3667d2e45e admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:19.833983 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1b22a-3bd0-409f-80a8-ca3667d2e45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:19.834256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1b22a-3bd0-409f-80a8-ca3667d2e45e admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:19.834985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1b22a-3bd0-409f-80a8-ca3667d2e45e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:19.835221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1b22a-3bd0-409f-80a8-ca3667d2e45e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:19.837085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1b22a-3bd0-409f-80a8-ca3667d2e45e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:19.853445 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 997/1986] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:19.854757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963147b0-8a39-4808-a990-702bc8ab97fb None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:19.855047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963147b0-8a39-4808-a990-702bc8ab97fb None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:19.855645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963147b0-8a39-4808-a990-702bc8ab97fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:19.880958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-963147b0-8a39-4808-a990-702bc8ab97fb None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:19.882232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-963147b0-8a39-4808-a990-702bc8ab97fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:19.894892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 998/1987] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:19.896671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963147b0-8a39-4808-a990-702bc8ab97fb None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:19.899927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-963147b0-8a39-4808-a990-702bc8ab97fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:19.915027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963147b0-8a39-4808-a990-702bc8ab97fb None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:19.919911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d325726-4454-4729-ae5e-573ee121c824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:19.920558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d325726-4454-4729-ae5e-573ee121c824 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:19.920837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d325726-4454-4729-ae5e-573ee121c824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:19.921112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d325726-4454-4729-ae5e-573ee121c824 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:19.921760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d325726-4454-4729-ae5e-573ee121c824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:19.923877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d325726-4454-4729-ae5e-573ee121c824 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:19.978122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 990/1988] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:19 2022] POST /identity/v3/users => generated 290 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:19.980659 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c8a42-d9ad-401b-b269-64058bc4da80 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:19.983189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ee0c8a42-d9ad-401b-b269-64058bc4da80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:20.020675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 999/1989] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:19 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:20.022246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d325726-4454-4729-ae5e-573ee121c824 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:20.112125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c8a42-d9ad-401b-b269-64058bc4da80 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:20.114941 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d325726-4454-4729-ae5e-573ee121c824 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:20.121483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80486bc6-d93f-47d8-b16e-50b9697b6426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:20.122125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80486bc6-d93f-47d8-b16e-50b9697b6426 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.122410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80486bc6-d93f-47d8-b16e-50b9697b6426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.122719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80486bc6-d93f-47d8-b16e-50b9697b6426 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.123658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80486bc6-d93f-47d8-b16e-50b9697b6426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:20.125465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80486bc6-d93f-47d8-b16e-50b9697b6426 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:20.146703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1000/1990] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:20.146703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80486bc6-d93f-47d8-b16e-50b9697b6426 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:20.155453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80486bc6-d93f-47d8-b16e-50b9697b6426 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:20.161393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84a14b-475a-4e8e-9ff1-a5d9573c118c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:20.161749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a14b-475a-4e8e-9ff1-a5d9573c118c admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.162043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a14b-475a-4e8e-9ff1-a5d9573c118c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.162323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84a14b-475a-4e8e-9ff1-a5d9573c118c admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.163066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84a14b-475a-4e8e-9ff1-a5d9573c118c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:20.165091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84a14b-475a-4e8e-9ff1-a5d9573c118c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:20.188263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591cc86-4921-43e9-86b7-0772ce592b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:20.188898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591cc86-4921-43e9-86b7-0772ce592b8f service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.189182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591cc86-4921-43e9-86b7-0772ce592b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.189459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591cc86-4921-43e9-86b7-0772ce592b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.201100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591cc86-4921-43e9-86b7-0772ce592b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:20.203221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591cc86-4921-43e9-86b7-0772ce592b8f service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:20.214394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 991/1991] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:20.215337 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591cc86-4921-43e9-86b7-0772ce592b8f service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:20.226738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591cc86-4921-43e9-86b7-0772ce592b8f service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:20.235919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132a805-58ca-445d-bd8b-5d34c479bd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:20.236604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a805-58ca-445d-bd8b-5d34c479bd6d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.236888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a805-58ca-445d-bd8b-5d34c479bd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.237173 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a805-58ca-445d-bd8b-5d34c479bd6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.237825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132a805-58ca-445d-bd8b-5d34c479bd6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:20.239457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132a805-58ca-445d-bd8b-5d34c479bd6d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:20.252357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 992/1992] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:20.258346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132a805-58ca-445d-bd8b-5d34c479bd6d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:20.276664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132a805-58ca-445d-bd8b-5d34c479bd6d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:20.285616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1001/1993] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:20 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:20.285858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6cdf1a-a52e-4df6-b4f8-d8612474f6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:20.286460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cdf1a-a52e-4df6-b4f8-d8612474f6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.286882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cdf1a-a52e-4df6-b4f8-d8612474f6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.287041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6cdf1a-a52e-4df6-b4f8-d8612474f6a5 admin admin] PATH_INFO: `/v3/projects/0ae858c183c04be58a7fc3ccfd0b8649/users/8eb38c1cf4c14eed9d51e0b649cbf993/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.293177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84a14b-475a-4e8e-9ff1-a5d9573c118c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:20.306396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84a14b-475a-4e8e-9ff1-a5d9573c118c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:20.309672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cdf1a-a52e-4df6-b4f8-d8612474f6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae858c183c04be58a7fc3ccfd0b8649, user_id=8eb38c1cf4c14eed9d51e0b649cbf993, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:20.313267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6cdf1a-a52e-4df6-b4f8-d8612474f6a5 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:20.320319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af9115-cf8c-4484-97f1-9e92a112fbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:20.321432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af9115-cf8c-4484-97f1-9e92a112fbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.321895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af9115-cf8c-4484-97f1-9e92a112fbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.322481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af9115-cf8c-4484-97f1-9e92a112fbd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.323213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af9115-cf8c-4484-97f1-9e92a112fbd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:20.325673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af9115-cf8c-4484-97f1-9e92a112fbd6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:20.340510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1002/1994] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:20.344878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af9115-cf8c-4484-97f1-9e92a112fbd6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:20.356158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af9115-cf8c-4484-97f1-9e92a112fbd6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:20.361555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dced8-a8c1-4cf9-8982-a5220d647695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:20.362191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dced8-a8c1-4cf9-8982-a5220d647695 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.362467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dced8-a8c1-4cf9-8982-a5220d647695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.362923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7dced8-a8c1-4cf9-8982-a5220d647695 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.363632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dced8-a8c1-4cf9-8982-a5220d647695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:20.365342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7dced8-a8c1-4cf9-8982-a5220d647695 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:20.379714 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1003/1995] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:20.385169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dced8-a8c1-4cf9-8982-a5220d647695 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:20.399745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7dced8-a8c1-4cf9-8982-a5220d647695 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:20.405911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f85ad-de99-4f00-aee7-edae69b1546b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:20.406613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f85ad-de99-4f00-aee7-edae69b1546b admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.406901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f85ad-de99-4f00-aee7-edae69b1546b admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.407181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f85ad-de99-4f00-aee7-edae69b1546b admin admin] PATH_INFO: `/v3/projects/1de2dfd53d724e6c93ce674a21aa9abf/users/bdc8f6e160aa46ddb52dbad0f41b7b5a/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.424229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f85ad-de99-4f00-aee7-edae69b1546b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de2dfd53d724e6c93ce674a21aa9abf, user_id=bdc8f6e160aa46ddb52dbad0f41b7b5a, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:20.427310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f85ad-de99-4f00-aee7-edae69b1546b admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:20.524772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 993/1996] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:20 2022] PUT /identity/v3/projects/0ae858c183c04be58a7fc3ccfd0b8649/users/8eb38c1cf4c14eed9d51e0b649cbf993/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:20.536299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f251be-91f8-44b8-acfe-179ebc6c7c5f None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.536658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f251be-91f8-44b8-acfe-179ebc6c7c5f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.536942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f251be-91f8-44b8-acfe-179ebc6c7c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.569308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e8f251be-91f8-44b8-acfe-179ebc6c7c5f None None] MFA Rules not processed for user `8eb38c1cf4c14eed9d51e0b649cbf993`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:20.570532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1004/1997] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:20 2022] PUT /identity/v3/projects/1de2dfd53d724e6c93ce674a21aa9abf/users/bdc8f6e160aa46ddb52dbad0f41b7b5a/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:20.570698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e8f251be-91f8-44b8-acfe-179ebc6c7c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:20.583370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c89d2-292f-4c02-9a59-5bfc27446f6d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.583693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c89d2-292f-4c02-9a59-5bfc27446f6d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.583972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c89d2-292f-4c02-9a59-5bfc27446f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.620939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-eb1c89d2-292f-4c02-9a59-5bfc27446f6d None None] MFA Rules not processed for user `bdc8f6e160aa46ddb52dbad0f41b7b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:20.623092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-eb1c89d2-292f-4c02-9a59-5bfc27446f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:20.655968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 994/1998] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:20.661544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f251be-91f8-44b8-acfe-179ebc6c7c5f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:20.777168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1005/1999] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:20.805447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c89d2-292f-4c02-9a59-5bfc27446f6d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:20.815114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-eb1c89d2-292f-4c02-9a59-5bfc27446f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:20.821692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f251be-91f8-44b8-acfe-179ebc6c7c5f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:20.833458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a8452-3147-4a40-8dbc-47132973990c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:20.834617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a8452-3147-4a40-8dbc-47132973990c admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.834617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a8452-3147-4a40-8dbc-47132973990c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.836372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a8452-3147-4a40-8dbc-47132973990c admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.838162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a8452-3147-4a40-8dbc-47132973990c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:20.845794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a8452-3147-4a40-8dbc-47132973990c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:20.888993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:20.917232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c89d2-292f-4c02-9a59-5bfc27446f6d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:20.922345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cc9d5-cd33-4913-8f76-a205d72abd01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:20.923077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc9d5-cd33-4913-8f76-a205d72abd01 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:20.923375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc9d5-cd33-4913-8f76-a205d72abd01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:20.923715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561cc9d5-cd33-4913-8f76-a205d72abd01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:20.935757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cc9d5-cd33-4913-8f76-a205d72abd01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:20.937981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561cc9d5-cd33-4913-8f76-a205d72abd01 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:20.955300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1006/2000] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:20.989042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cc9d5-cd33-4913-8f76-a205d72abd01 service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:21.001435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561cc9d5-cd33-4913-8f76-a205d72abd01 service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:21.006711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:21.007363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:21.007686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:21.007966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] PATH_INFO: `/v3/users/bdc8f6e160aa46ddb52dbad0f41b7b5a` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:21.009188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc8f6e160aa46ddb52dbad0f41b7b5a)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:21.011174 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:21.058864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 995/2001] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:20 2022] POST /identity/v3/projects => generated 379 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:31:21.062953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a8452-3147-4a40-8dbc-47132973990c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:21.136728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a8452-3147-4a40-8dbc-47132973990c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:21.143158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f986648-307a-4655-8cd7-7b2ad45f3474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:21.143965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f986648-307a-4655-8cd7-7b2ad45f3474 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:21.144462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f986648-307a-4655-8cd7-7b2ad45f3474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:21.144928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f986648-307a-4655-8cd7-7b2ad45f3474 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:21.145974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f986648-307a-4655-8cd7-7b2ad45f3474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:21.148122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:21.148824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f986648-307a-4655-8cd7-7b2ad45f3474 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:21.416359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 996/2002] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:21 2022] POST /identity/v3/users => generated 290 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:31:21.418008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f986648-307a-4655-8cd7-7b2ad45f3474 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:21.420976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6f986648-307a-4655-8cd7-7b2ad45f3474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:21.437325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f986648-307a-4655-8cd7-7b2ad45f3474 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:21.444995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:21.446105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:21.446590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:21.447089 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] PATH_INFO: `/v3/projects/bce7263e4aee46beab417b720fefa59d` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:21.449361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce7263e4aee46beab417b720fefa59d)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:21.453848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:21.494065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc8f6e160aa46ddb52dbad0f41b7b5a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:21.495230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc8f6e160aa46ddb52dbad0f41b7b5a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:21.496429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc8f6e160aa46ddb52dbad0f41b7b5a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:21.508253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc8f6e160aa46ddb52dbad0f41b7b5a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:21.531202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc8f6e160aa46ddb52dbad0f41b7b5a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:21.725730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1007/2003] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:20 2022] DELETE /identity/v3/users/bdc8f6e160aa46ddb52dbad0f41b7b5a => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:21.726845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:21.730281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:21.745445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce7263e4aee46beab417b720fefa59d', 'request_id': 'req-6825280c-8cd6-4832-b3f6-879085904aa1'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:21.754604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce7263e4aee46beab417b720fefa59d', 'request_id': 'req-6825280c-8cd6-4832-b3f6-879085904aa1'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:21.807186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493ae6f-b59d-461e-a901-ba78d8b7862a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:21.813263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b515c3-1364-4c65-ad8d-7640ca2cb13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:21.813983 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b515c3-1364-4c65-ad8d-7640ca2cb13e admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:21.814280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b515c3-1364-4c65-ad8d-7640ca2cb13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:21.814568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b515c3-1364-4c65-ad8d-7640ca2cb13e admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:21.815295 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b515c3-1364-4c65-ad8d-7640ca2cb13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:21.817101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b515c3-1364-4c65-ad8d-7640ca2cb13e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:21.830361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1008/2004] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:21.833930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b314-6db4-4392-9a4d-c03d6508119b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:21.834438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b314-6db4-4392-9a4d-c03d6508119b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:21.834979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b314-6db4-4392-9a4d-c03d6508119b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:21.857721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b686b314-6db4-4392-9a4d-c03d6508119b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:21.859025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b686b314-6db4-4392-9a4d-c03d6508119b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:21.867791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1009/2005] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:21.868742 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b314-6db4-4392-9a4d-c03d6508119b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:21.881766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b314-6db4-4392-9a4d-c03d6508119b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:21.886570 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf729d0c-26c2-4904-b51e-4a52b1c1576f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:21.887226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf729d0c-26c2-4904-b51e-4a52b1c1576f admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:21.887500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf729d0c-26c2-4904-b51e-4a52b1c1576f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:21.887802 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf729d0c-26c2-4904-b51e-4a52b1c1576f admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:21.888457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf729d0c-26c2-4904-b51e-4a52b1c1576f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:21.899016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf729d0c-26c2-4904-b51e-4a52b1c1576f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:21.902281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1010/2006] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:21.903196 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf729d0c-26c2-4904-b51e-4a52b1c1576f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:21.905246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bf729d0c-26c2-4904-b51e-4a52b1c1576f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:21.935166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce7263e4aee46beab417b720fefa59d', 'request_id': 'req-6825280c-8cd6-4832-b3f6-879085904aa1'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:21.950820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] The token cache is being invalidate because project bce7263e4aee46beab417b720fefa59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:21.951116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:21.960022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:21.960022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 997/2007] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:21 2022] DELETE /identity/v3/projects/bce7263e4aee46beab417b720fefa59d => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:21.960022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:21.960022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:21.987747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf729d0c-26c2-4904-b51e-4a52b1c1576f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:21.994469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23f44a-d2df-41d0-a8d7-599d6159208d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:21.995530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23f44a-d2df-41d0-a8d7-599d6159208d service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:21.996067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23f44a-d2df-41d0-a8d7-599d6159208d service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:21.996768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23f44a-d2df-41d0-a8d7-599d6159208d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:22.000184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ea23f44a-d2df-41d0-a8d7-599d6159208d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:22.024015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23f44a-d2df-41d0-a8d7-599d6159208d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:22.026003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23f44a-d2df-41d0-a8d7-599d6159208d service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:22.037865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1011/2008] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:22.041264 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23f44a-d2df-41d0-a8d7-599d6159208d service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:22.044441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ea23f44a-d2df-41d0-a8d7-599d6159208d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:22.062106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825280c-8cd6-4832-b3f6-879085904aa1 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:22.070855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc45e75-b524-45f7-a143-bae7e5a0479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:22.074300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23f44a-d2df-41d0-a8d7-599d6159208d service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:22.075913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc45e75-b524-45f7-a143-bae7e5a0479e admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:22.077079 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc45e75-b524-45f7-a143-bae7e5a0479e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:22.077519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc45e75-b524-45f7-a143-bae7e5a0479e admin admin] PATH_INFO: `/v3/projects/614af2c2a5c8452cb9e70eb607cb5d4a/users/8313f22a39424320870259c32c2035cb/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:22.104226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:22.108350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:22.108720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:22.109330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] PATH_INFO: `/v3/users/a85961ae47944da09f5c2bcc80cb271a` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:22.115943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] RBAC: Authorizing `identity:delete_user(user_id=a85961ae47944da09f5c2bcc80cb271a)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:22.120727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc45e75-b524-45f7-a143-bae7e5a0479e admin admin] RBAC: Authorizing `identity:create_grant(project_id=614af2c2a5c8452cb9e70eb607cb5d4a, user_id=8313f22a39424320870259c32c2035cb, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:22.124835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc45e75-b524-45f7-a143-bae7e5a0479e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:22.126080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:22.272458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 998/2009] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:21 2022] PUT /identity/v3/projects/614af2c2a5c8452cb9e70eb607cb5d4a/users/8313f22a39424320870259c32c2035cb/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:22.289970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f76cea-85a9-4f4d-b92e-7779ebc78ad9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:22.289970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f76cea-85a9-4f4d-b92e-7779ebc78ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:22.292010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f76cea-85a9-4f4d-b92e-7779ebc78ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:22.330569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-18f76cea-85a9-4f4d-b92e-7779ebc78ad9 None None] MFA Rules not processed for user `8313f22a39424320870259c32c2035cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:22.332206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-18f76cea-85a9-4f4d-b92e-7779ebc78ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:22.418054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a85961ae47944da09f5c2bcc80cb271a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:22.419520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a85961ae47944da09f5c2bcc80cb271a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:22.421275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a85961ae47944da09f5c2bcc80cb271a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:22.437255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a85961ae47944da09f5c2bcc80cb271a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:22.457307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 999/2010] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:22.463130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f76cea-85a9-4f4d-b92e-7779ebc78ad9 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:22.480963 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-18f76cea-85a9-4f4d-b92e-7779ebc78ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:22.481187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a85961ae47944da09f5c2bcc80cb271a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:22.548192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f76cea-85a9-4f4d-b92e-7779ebc78ad9 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:22.553753 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe4c25-a196-4b4a-929a-4914d4d4bfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:22.554395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe4c25-a196-4b4a-929a-4914d4d4bfc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:22.554685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe4c25-a196-4b4a-929a-4914d4d4bfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:22.555047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe4c25-a196-4b4a-929a-4914d4d4bfc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:22.555655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe4c25-a196-4b4a-929a-4914d4d4bfc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:22.557750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe4c25-a196-4b4a-929a-4914d4d4bfc8 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:22.657859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1012/2011] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:22 2022] DELETE /identity/v3/users/a85961ae47944da09f5c2bcc80cb271a => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:22.716360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:22.716360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:22.761393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1000/2012] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:22 2022] POST /identity/v3/projects => generated 377 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:31:22.770993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe4c25-a196-4b4a-929a-4914d4d4bfc8 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:22.864117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:22.920723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3f0a0-a5d7-4d46-bfa2-ba44e0ad57de admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:22.968130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe4c25-a196-4b4a-929a-4914d4d4bfc8 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:22.976215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06e90b-6894-4fd4-ba03-9c48c40d8d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:22.977194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06e90b-6894-4fd4-ba03-9c48c40d8d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:22.977936 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06e90b-6894-4fd4-ba03-9c48c40d8d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:22.978435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06e90b-6894-4fd4-ba03-9c48c40d8d53 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:22.979540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06e90b-6894-4fd4-ba03-9c48c40d8d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:22.983019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06e90b-6894-4fd4-ba03-9c48c40d8d53 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:22.996930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:23.016404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:23.017180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:23.017557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:23.018058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] PATH_INFO: `/v3/projects/1de2dfd53d724e6c93ce674a21aa9abf` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:23.019933 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1de2dfd53d724e6c93ce674a21aa9abf)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:23.022633 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:23.150706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1001/2013] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:22 2022] POST /identity/v3/users => generated 295 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:23.156243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06e90b-6894-4fd4-ba03-9c48c40d8d53 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:23.169658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06e90b-6894-4fd4-ba03-9c48c40d8d53 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:23.176150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0551831-f59e-4cc4-9396-49603ad45d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:23.176976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0551831-f59e-4cc4-9396-49603ad45d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:23.177327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0551831-f59e-4cc4-9396-49603ad45d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:23.177777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0551831-f59e-4cc4-9396-49603ad45d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:23.178593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0551831-f59e-4cc4-9396-49603ad45d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:23.181055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0551831-f59e-4cc4-9396-49603ad45d7d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:23.196251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1002/2014] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:23.200608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0551831-f59e-4cc4-9396-49603ad45d7d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:23.215940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0551831-f59e-4cc4-9396-49603ad45d7d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:23.253499 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1de2dfd53d724e6c93ce674a21aa9abf', 'request_id': 'req-ad07a435-8832-40c2-96e1-7927b60ffc1c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:23.265680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1de2dfd53d724e6c93ce674a21aa9abf', 'request_id': 'req-ad07a435-8832-40c2-96e1-7927b60ffc1c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:23.332945 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be84e466-72bd-4092-b4d7-69a45226c5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:23.333859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84e466-72bd-4092-b4d7-69a45226c5f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:23.334305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84e466-72bd-4092-b4d7-69a45226c5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:23.334852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84e466-72bd-4092-b4d7-69a45226c5f8 admin admin] PATH_INFO: `/v3/projects/f6556ad62de2429b8f98b3f6432192fd/users/7d605fa3f15e48e89ec249dfb6c4863b/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:23.358035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84e466-72bd-4092-b4d7-69a45226c5f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6556ad62de2429b8f98b3f6432192fd, user_id=7d605fa3f15e48e89ec249dfb6c4863b, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:23.361389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be84e466-72bd-4092-b4d7-69a45226c5f8 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:23.377130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1de2dfd53d724e6c93ce674a21aa9abf', 'request_id': 'req-ad07a435-8832-40c2-96e1-7927b60ffc1c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:23.398878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] The token cache is being invalidate because project 1de2dfd53d724e6c93ce674a21aa9abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:23.399419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:23.400697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ad07a435-8832-40c2-96e1-7927b60ffc1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:23.403386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1013/2015] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:22 2022] DELETE /identity/v3/projects/1de2dfd53d724e6c93ce674a21aa9abf => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:23.412144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70ccaf-1dfd-4542-bb27-7ae4cd6d8d39 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:23.412659 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70ccaf-1dfd-4542-bb27-7ae4cd6d8d39 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:23.413172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70ccaf-1dfd-4542-bb27-7ae4cd6d8d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:23.452050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9e70ccaf-1dfd-4542-bb27-7ae4cd6d8d39 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:23.453895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9e70ccaf-1dfd-4542-bb27-7ae4cd6d8d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:23.476811 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1014/2016] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:23.478141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70ccaf-1dfd-4542-bb27-7ae4cd6d8d39 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:23.482093 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1003/2017] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:23 2022] PUT /identity/v3/projects/f6556ad62de2429b8f98b3f6432192fd/users/7d605fa3f15e48e89ec249dfb6c4863b/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:23.482190 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9e70ccaf-1dfd-4542-bb27-7ae4cd6d8d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:23.482365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ffb1f-1e5a-48c5-a18a-f150c6c6c84a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:23.483095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ffb1f-1e5a-48c5-a18a-f150c6c6c84a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:23.483909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ffb1f-1e5a-48c5-a18a-f150c6c6c84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:23.500661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70ccaf-1dfd-4542-bb27-7ae4cd6d8d39 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:23.516437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-081ffb1f-1e5a-48c5-a18a-f150c6c6c84a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:23.518048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-081ffb1f-1e5a-48c5-a18a-f150c6c6c84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:23.604783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:23.605910 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:23.606211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:23.606644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] PATH_INFO: `/v3/projects/3c7a8dbc4e7a42f0822dfdc780fa1172` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:23.608903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7a8dbc4e7a42f0822dfdc780fa1172)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:23.611919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:23.613420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1004/2018] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:23.614727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ffb1f-1e5a-48c5-a18a-f150c6c6c84a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:23.616869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-081ffb1f-1e5a-48c5-a18a-f150c6c6c84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:23.638135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ffb1f-1e5a-48c5-a18a-f150c6c6c84a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:23.644792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce6b88-af85-4556-9bb5-02eb8c2ae06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:23.645467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce6b88-af85-4556-9bb5-02eb8c2ae06c admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:23.645761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce6b88-af85-4556-9bb5-02eb8c2ae06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:23.646040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce6b88-af85-4556-9bb5-02eb8c2ae06c admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:23.647010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce6b88-af85-4556-9bb5-02eb8c2ae06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:23.649789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce6b88-af85-4556-9bb5-02eb8c2ae06c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:23.673069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1005/2019] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:23.675679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce6b88-af85-4556-9bb5-02eb8c2ae06c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:23.694502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce6b88-af85-4556-9bb5-02eb8c2ae06c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:23.704052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:23.708494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82069b83-f1b2-4013-a6c5-01e99d79d093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:23.708494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82069b83-f1b2-4013-a6c5-01e99d79d093 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:23.708494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82069b83-f1b2-4013-a6c5-01e99d79d093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:23.708494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82069b83-f1b2-4013-a6c5-01e99d79d093 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:23.710271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82069b83-f1b2-4013-a6c5-01e99d79d093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:23.711044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82069b83-f1b2-4013-a6c5-01e99d79d093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:23.714841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82069b83-f1b2-4013-a6c5-01e99d79d093 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:23.761256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1006/2020] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:23.762897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82069b83-f1b2-4013-a6c5-01e99d79d093 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:23.767425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7a8dbc4e7a42f0822dfdc780fa1172', 'request_id': 'req-0c46f90b-4460-47f8-be04-bfe033a8cffb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:23.778863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82069b83-f1b2-4013-a6c5-01e99d79d093 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:23.783335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7a8dbc4e7a42f0822dfdc780fa1172', 'request_id': 'req-0c46f90b-4460-47f8-be04-bfe033a8cffb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:23.879877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc8033-a885-4110-ab2a-9bcc184524be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:23.880730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc8033-a885-4110-ab2a-9bcc184524be admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:23.881010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc8033-a885-4110-ab2a-9bcc184524be admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:23.881282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc8033-a885-4110-ab2a-9bcc184524be admin admin] PATH_INFO: `/v3/projects/f6556ad62de2429b8f98b3f6432192fd/users/7d605fa3f15e48e89ec249dfb6c4863b/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:23.883912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc8033-a885-4110-ab2a-9bcc184524be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6556ad62de2429b8f98b3f6432192fd, user_id=7d605fa3f15e48e89ec249dfb6c4863b, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:23.886692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc8033-a885-4110-ab2a-9bcc184524be admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:23.887346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:23.902195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1007/2021] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:23 2022] PUT /identity/v3/projects/f6556ad62de2429b8f98b3f6432192fd/users/7d605fa3f15e48e89ec249dfb6c4863b/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:23.913159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc8033-a885-4110-ab2a-9bcc184524be admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:23.924394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7a8dbc4e7a42f0822dfdc780fa1172', 'request_id': 'req-0c46f90b-4460-47f8-be04-bfe033a8cffb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:23.947985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] The token cache is being invalidate because project 3c7a8dbc4e7a42f0822dfdc780fa1172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:23.948488 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:23.949281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0c46f90b-4460-47f8-be04-bfe033a8cffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:23.951783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1015/2022] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:23 2022] DELETE /identity/v3/projects/3c7a8dbc4e7a42f0822dfdc780fa1172 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:23.952719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459bce1-6d01-42b2-8887-bd2cd3292d3d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:23.952996 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459bce1-6d01-42b2-8887-bd2cd3292d3d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:23.953407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459bce1-6d01-42b2-8887-bd2cd3292d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.012928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6459bce1-6d01-42b2-8887-bd2cd3292d3d None None] MFA Rules not processed for user `7d605fa3f15e48e89ec249dfb6c4863b`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:24.015076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6459bce1-6d01-42b2-8887-bd2cd3292d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:24.045991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:24.059585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc8033-a885-4110-ab2a-9bcc184524be admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:24.072604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67bc6a-1b41-4964-a45d-6401c9c74d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:24.074823 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67bc6a-1b41-4964-a45d-6401c9c74d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:24.076474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67bc6a-1b41-4964-a45d-6401c9c74d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:24.077373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67bc6a-1b41-4964-a45d-6401c9c74d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.079254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67bc6a-1b41-4964-a45d-6401c9c74d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:24.086786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67bc6a-1b41-4964-a45d-6401c9c74d73 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:24.113623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:24.147816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1016/2023] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:24.150363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbb75e-14ae-48dd-bb50-bf1f5de0230f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:24.150890 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbb75e-14ae-48dd-bb50-bf1f5de0230f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:24.151393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbb75e-14ae-48dd-bb50-bf1f5de0230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.186906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-88dbb75e-14ae-48dd-bb50-bf1f5de0230f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:24.189210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-88dbb75e-14ae-48dd-bb50-bf1f5de0230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:24.218222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1017/2024] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:24.218222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbb75e-14ae-48dd-bb50-bf1f5de0230f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:24.218222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-88dbb75e-14ae-48dd-bb50-bf1f5de0230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:24.301436 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1008/2025] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:23 2022] POST /identity/v3/projects => generated 383 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:24.306688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f82de4-33a8-49df-a4fc-d684db37157d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:24.307257 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f82de4-33a8-49df-a4fc-d684db37157d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:24.307744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f82de4-33a8-49df-a4fc-d684db37157d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.320399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbb75e-14ae-48dd-bb50-bf1f5de0230f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:24.379492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-47f82de4-33a8-49df-a4fc-d684db37157d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:24.381995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-47f82de4-33a8-49df-a4fc-d684db37157d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:24.461984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e216c39-b776-4b65-900f-1bee1eb6f1dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:24.464142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e216c39-b776-4b65-900f-1bee1eb6f1dc service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:24.464707 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e216c39-b776-4b65-900f-1bee1eb6f1dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:24.465132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e216c39-b776-4b65-900f-1bee1eb6f1dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.557698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1009/2026] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:24.557698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f82de4-33a8-49df-a4fc-d684db37157d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:24.558449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-47f82de4-33a8-49df-a4fc-d684db37157d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:24.577179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f82de4-33a8-49df-a4fc-d684db37157d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:24.584709 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:24.589369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e29a92-e5ca-4d18-a3ee-6b1c3bce100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:24.590017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e29a92-e5ca-4d18-a3ee-6b1c3bce100e admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:24.590300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e29a92-e5ca-4d18-a3ee-6b1c3bce100e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:24.590876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e29a92-e5ca-4d18-a3ee-6b1c3bce100e admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.592914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e29a92-e5ca-4d18-a3ee-6b1c3bce100e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:24.594665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e29a92-e5ca-4d18-a3ee-6b1c3bce100e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:24.618595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e216c39-b776-4b65-900f-1bee1eb6f1dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:24.620692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e216c39-b776-4b65-900f-1bee1eb6f1dc service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:24.634370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1018/2027] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:24.635901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e216c39-b776-4b65-900f-1bee1eb6f1dc service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:24.651658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e216c39-b776-4b65-900f-1bee1eb6f1dc service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:24.660461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0180-c2ee-49a4-baf9-ff90d87bc852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:24.661357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0180-c2ee-49a4-baf9-ff90d87bc852 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:24.661790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0180-c2ee-49a4-baf9-ff90d87bc852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:24.662223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd0180-c2ee-49a4-baf9-ff90d87bc852 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.663142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0180-c2ee-49a4-baf9-ff90d87bc852 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:24.663714 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0180-c2ee-49a4-baf9-ff90d87bc852 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:24.666899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd0180-c2ee-49a4-baf9-ff90d87bc852 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:24.688859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1010/2028] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:24 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:24.691866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1019/2029] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:24.696773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e29a92-e5ca-4d18-a3ee-6b1c3bce100e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:24.699614 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0180-c2ee-49a4-baf9-ff90d87bc852 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:24.704307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:24.712374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e29a92-e5ca-4d18-a3ee-6b1c3bce100e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:24.722330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30ea64-f06d-43b4-9bc3-d8fbe4e37827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:24.723174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30ea64-f06d-43b4-9bc3-d8fbe4e37827 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:24.723597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30ea64-f06d-43b4-9bc3-d8fbe4e37827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:24.724213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30ea64-f06d-43b4-9bc3-d8fbe4e37827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.725119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30ea64-f06d-43b4-9bc3-d8fbe4e37827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:24.727451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30ea64-f06d-43b4-9bc3-d8fbe4e37827 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:24.733112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd0180-c2ee-49a4-baf9-ff90d87bc852 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:24.740801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1011/2030] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:24.748814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30ea64-f06d-43b4-9bc3-d8fbe4e37827 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:24.749188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c29cbb-35a5-4b24-8876-8f5798c170a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:24.749938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c29cbb-35a5-4b24-8876-8f5798c170a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:24.750303 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c29cbb-35a5-4b24-8876-8f5798c170a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:24.750663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c29cbb-35a5-4b24-8876-8f5798c170a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.751420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c29cbb-35a5-4b24-8876-8f5798c170a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:24.754355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c29cbb-35a5-4b24-8876-8f5798c170a3 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:24.771952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30ea64-f06d-43b4-9bc3-d8fbe4e37827 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:24.781971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae51703-82a0-488f-b032-df51525f9a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:24.783135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae51703-82a0-488f-b032-df51525f9a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:24.784605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae51703-82a0-488f-b032-df51525f9a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:24.784605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae51703-82a0-488f-b032-df51525f9a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.785728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae51703-82a0-488f-b032-df51525f9a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:24.789073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae51703-82a0-488f-b032-df51525f9a3e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:24.812295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1012/2031] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:24.814345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae51703-82a0-488f-b032-df51525f9a3e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:24.817514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-bae51703-82a0-488f-b032-df51525f9a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:24.836725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae51703-82a0-488f-b032-df51525f9a3e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:24.841822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55799a-4eff-496b-926b-84c2181ecbd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:24.842636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55799a-4eff-496b-926b-84c2181ecbd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:24.842917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55799a-4eff-496b-926b-84c2181ecbd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:24.843199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55799a-4eff-496b-926b-84c2181ecbd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:24.845711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9b55799a-4eff-496b-926b-84c2181ecbd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:24.872279 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1020/2032] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:24 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:24.874475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c29cbb-35a5-4b24-8876-8f5798c170a3 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:24.939631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55799a-4eff-496b-926b-84c2181ecbd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:24.941285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55799a-4eff-496b-926b-84c2181ecbd4 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:24.953987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1013/2033] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:24.955445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55799a-4eff-496b-926b-84c2181ecbd4 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:24.976154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55799a-4eff-496b-926b-84c2181ecbd4 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:25.026238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c29cbb-35a5-4b24-8876-8f5798c170a3 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:25.035844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d39ac-4b2f-4e23-97e1-7a3e3afbf01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:25.037164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d39ac-4b2f-4e23-97e1-7a3e3afbf01f admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:25.037653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d39ac-4b2f-4e23-97e1-7a3e3afbf01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:25.039440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d39ac-4b2f-4e23-97e1-7a3e3afbf01f admin admin] PATH_INFO: `/v3/projects/c2a148e97ebd4a88b83747b5ef0773e8/users/4f0a1e4e0d364927858eddbc38d13404/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:25.072501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d39ac-4b2f-4e23-97e1-7a3e3afbf01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2a148e97ebd4a88b83747b5ef0773e8, user_id=4f0a1e4e0d364927858eddbc38d13404, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:25.078775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d39ac-4b2f-4e23-97e1-7a3e3afbf01f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:25.129120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1558e-616c-47b9-baad-18524abcb8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:25.130451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1558e-616c-47b9-baad-18524abcb8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:25.130954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1558e-616c-47b9-baad-18524abcb8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:25.131454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1558e-616c-47b9-baad-18524abcb8f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:25.132711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1558e-616c-47b9-baad-18524abcb8f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:25.136800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1558e-616c-47b9-baad-18524abcb8f2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:25.224691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1021/2034] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:24 2022] PUT /identity/v3/projects/c2a148e97ebd4a88b83747b5ef0773e8/users/4f0a1e4e0d364927858eddbc38d13404/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:25.243508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cbf79-637a-4f2d-a309-953b65e91570 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:25.244046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cbf79-637a-4f2d-a309-953b65e91570 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:25.244744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cbf79-637a-4f2d-a309-953b65e91570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:25.267516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1014/2035] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:24 2022] POST /identity/v3/users => generated 287 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:25.272330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1558e-616c-47b9-baad-18524abcb8f2 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:25.290430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1558e-616c-47b9-baad-18524abcb8f2 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:25.308484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-a64cbf79-637a-4f2d-a309-953b65e91570 None None] MFA Rules not processed for user `4f0a1e4e0d364927858eddbc38d13404`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:25.310128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a64cbf79-637a-4f2d-a309-953b65e91570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:25.394715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1b03f-266f-4676-bc3d-f4d160a064b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:25.395387 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b03f-266f-4676-bc3d-f4d160a064b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:25.395725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b03f-266f-4676-bc3d-f4d160a064b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:25.396008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b03f-266f-4676-bc3d-f4d160a064b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:25.396773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1b03f-266f-4676-bc3d-f4d160a064b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:25.398864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1b03f-266f-4676-bc3d-f4d160a064b4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:25.426099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1015/2036] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:25.427158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1b03f-266f-4676-bc3d-f4d160a064b4 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:25.444649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1b03f-266f-4676-bc3d-f4d160a064b4 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:25.453368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a6be6-c35f-4457-8568-c5a5dcb4590f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:25.454305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a6be6-c35f-4457-8568-c5a5dcb4590f admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:25.454555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a6be6-c35f-4457-8568-c5a5dcb4590f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:25.454834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a6be6-c35f-4457-8568-c5a5dcb4590f admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:25.455623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a6be6-c35f-4457-8568-c5a5dcb4590f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:25.459613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a6be6-c35f-4457-8568-c5a5dcb4590f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:25.468258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1022/2037] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:25.479815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1016/2038] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:25.483178 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cbf79-637a-4f2d-a309-953b65e91570 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:25.484574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a6be6-c35f-4457-8568-c5a5dcb4590f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:25.497608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cbf79-637a-4f2d-a309-953b65e91570 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:25.504821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3d078-c9c0-426e-8a45-85c07f14a46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:25.505754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3d078-c9c0-426e-8a45-85c07f14a46d admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:25.506276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3d078-c9c0-426e-8a45-85c07f14a46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:25.506579 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3d078-c9c0-426e-8a45-85c07f14a46d admin admin] PATH_INFO: `/v3/projects/79578cfa5efb4cb1ba9e926639641a55/users/35988e02d2f94c25a55a1c203e08e820/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:25.529833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3d078-c9c0-426e-8a45-85c07f14a46d admin admin] RBAC: Authorizing `identity:create_grant(project_id=79578cfa5efb4cb1ba9e926639641a55, user_id=35988e02d2f94c25a55a1c203e08e820, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:25.534650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3d078-c9c0-426e-8a45-85c07f14a46d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:25.593459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a6be6-c35f-4457-8568-c5a5dcb4590f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:25.601154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe81c0-91a5-4424-986e-09a31bccb812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:25.602966 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe81c0-91a5-4424-986e-09a31bccb812 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:25.603654 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe81c0-91a5-4424-986e-09a31bccb812 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:25.604265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe81c0-91a5-4424-986e-09a31bccb812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:25.622874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe81c0-91a5-4424-986e-09a31bccb812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:25.626134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe81c0-91a5-4424-986e-09a31bccb812 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:25.657816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1017/2039] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:25.664471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1023/2040] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:25 2022] PUT /identity/v3/projects/79578cfa5efb4cb1ba9e926639641a55/users/35988e02d2f94c25a55a1c203e08e820/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:25.675194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81377b-d021-41d0-b83c-8890e969ddb4 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:25.675853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81377b-d021-41d0-b83c-8890e969ddb4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:25.676607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81377b-d021-41d0-b83c-8890e969ddb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:25.738565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-bc81377b-d021-41d0-b83c-8890e969ddb4 None None] MFA Rules not processed for user `35988e02d2f94c25a55a1c203e08e820`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:25.742274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-bc81377b-d021-41d0-b83c-8890e969ddb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:25.776836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:25.903847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1018/2041] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:25.982470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:26.169257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3d078-c9c0-426e-8a45-85c07f14a46d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:26.258441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3d078-c9c0-426e-8a45-85c07f14a46d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:26.262688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015c38e-c602-4fce-a44c-a72681cf98c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:26.263349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015c38e-c602-4fce-a44c-a72681cf98c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:26.264320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015c38e-c602-4fce-a44c-a72681cf98c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:26.264590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015c38e-c602-4fce-a44c-a72681cf98c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:26.276354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015c38e-c602-4fce-a44c-a72681cf98c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:26.278053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015c38e-c602-4fce-a44c-a72681cf98c2 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:26.290952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1024/2042] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:26.324478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81377b-d021-41d0-b83c-8890e969ddb4 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:26.462694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015c38e-c602-4fce-a44c-a72681cf98c2 service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:26.464593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81377b-d021-41d0-b83c-8890e969ddb4 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:26.464936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6015c38e-c602-4fce-a44c-a72681cf98c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:26.470955 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:26.471867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:26.472384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:26.472745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] PATH_INFO: `/v3/users/35988e02d2f94c25a55a1c203e08e820` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:26.474932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35988e02d2f94c25a55a1c203e08e820)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:26.477686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:26.486377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015c38e-c602-4fce-a44c-a72681cf98c2 service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:26.492828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:26.493583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:26.493899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:26.494497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] PATH_INFO: `/v3/users/eaee6baf818a4c4481637cbfcc16d4fd` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:26.498902 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaee6baf818a4c4481637cbfcc16d4fd)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:26.500095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:26.893497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35988e02d2f94c25a55a1c203e08e820'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:26.897833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaee6baf818a4c4481637cbfcc16d4fd'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:26.900036 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaee6baf818a4c4481637cbfcc16d4fd'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:26.901498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaee6baf818a4c4481637cbfcc16d4fd'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:26.913762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaee6baf818a4c4481637cbfcc16d4fd'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:26.932938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6178870f-a4c5-422b-a4c1-2e276aa40da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaee6baf818a4c4481637cbfcc16d4fd'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:27.086078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35988e02d2f94c25a55a1c203e08e820'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:27.087191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35988e02d2f94c25a55a1c203e08e820'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:27.107302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35988e02d2f94c25a55a1c203e08e820'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:27.108693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5cda9649-c98e-4fa9-a849-7ae3944da06d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35988e02d2f94c25a55a1c203e08e820'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:27.126620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1019/2043] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:26 2022] DELETE /identity/v3/users/35988e02d2f94c25a55a1c203e08e820 => generated 0 bytes in 806 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:27.131822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e4078-a493-4be9-93ec-e2a9051b5aea None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:27.132194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e4078-a493-4be9-93ec-e2a9051b5aea None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:27.132573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e4078-a493-4be9-93ec-e2a9051b5aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:27.171635 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-dd8e4078-a493-4be9-93ec-e2a9051b5aea None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:27.171635 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-dd8e4078-a493-4be9-93ec-e2a9051b5aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:27.184668 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:27.243312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1025/2044] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:26 2022] DELETE /identity/v3/users/eaee6baf818a4c4481637cbfcc16d4fd => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:27.250743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5276a-8528-470b-9011-4bcff9f11f28 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:27.252591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5276a-8528-470b-9011-4bcff9f11f28 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:27.253716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5276a-8528-470b-9011-4bcff9f11f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:27.292134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1020/2045] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:27.300968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-01b5276a-8528-470b-9011-4bcff9f11f28 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:27.301889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-01b5276a-8528-470b-9011-4bcff9f11f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:27.309976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e4078-a493-4be9-93ec-e2a9051b5aea None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:27.312290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-dd8e4078-a493-4be9-93ec-e2a9051b5aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:27.525812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e4078-a493-4be9-93ec-e2a9051b5aea None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:27.568926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1026/2046] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:27.588383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5276a-8528-470b-9011-4bcff9f11f28 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:27.593490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-01b5276a-8528-470b-9011-4bcff9f11f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:27.668181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee353d86-2ce3-4648-b56c-5a7d00e4318c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:27.669187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee353d86-2ce3-4648-b56c-5a7d00e4318c service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:27.670365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee353d86-2ce3-4648-b56c-5a7d00e4318c service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:27.671763 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee353d86-2ce3-4648-b56c-5a7d00e4318c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:27.698794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee353d86-2ce3-4648-b56c-5a7d00e4318c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:27.704343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee353d86-2ce3-4648-b56c-5a7d00e4318c service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:27.737945 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1021/2047] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:27.766568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b5276a-8528-470b-9011-4bcff9f11f28 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:27.776099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade63b4-0f56-46a3-838c-b0024e46d711 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:27.777440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade63b4-0f56-46a3-838c-b0024e46d711 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:27.777946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade63b4-0f56-46a3-838c-b0024e46d711 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:27.779089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade63b4-0f56-46a3-838c-b0024e46d711 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:27.803376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade63b4-0f56-46a3-838c-b0024e46d711 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:27.806377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade63b4-0f56-46a3-838c-b0024e46d711 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:27.845995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1027/2048] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:27.910447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:28.609961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:28.827069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee353d86-2ce3-4648-b56c-5a7d00e4318c service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:28.828692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ee353d86-2ce3-4648-b56c-5a7d00e4318c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:28.894921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee353d86-2ce3-4648-b56c-5a7d00e4318c service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:28.909831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:28.912132 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:28.913175 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:28.914938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] PATH_INFO: `/v3/projects/6db6561fbd664d1b8d140517991c0f4e` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:28.920391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db6561fbd664d1b8d140517991c0f4e)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:28.924201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:29.153814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db6561fbd664d1b8d140517991c0f4e', 'request_id': 'req-c5201918-9a0b-4bcd-9d3b-eed194641339'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.164782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db6561fbd664d1b8d140517991c0f4e', 'request_id': 'req-c5201918-9a0b-4bcd-9d3b-eed194641339'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.185338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade63b4-0f56-46a3-838c-b0024e46d711 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:29.187638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-aade63b4-0f56-46a3-838c-b0024e46d711 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:29.207221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade63b4-0f56-46a3-838c-b0024e46d711 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:29.315621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db6561fbd664d1b8d140517991c0f4e', 'request_id': 'req-c5201918-9a0b-4bcd-9d3b-eed194641339'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.352555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] The token cache is being invalidate because project 6db6561fbd664d1b8d140517991c0f4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:29.353865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.355665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.364354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1022/2049] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:28 2022] DELETE /identity/v3/projects/6db6561fbd664d1b8d140517991c0f4e => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:29.366353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:29.369152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:29.372328 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:29.374312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:29.375109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:29.375807 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] PATH_INFO: `/v3/projects/79578cfa5efb4cb1ba9e926639641a55` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:29.378142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] RBAC: Authorizing `identity:delete_project(project_id=79578cfa5efb4cb1ba9e926639641a55)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:29.382376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:29.394007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5201918-9a0b-4bcd-9d3b-eed194641339 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:29.401208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:29.402145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:29.402598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:29.402966 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] PATH_INFO: `/v3/users/7324b8c749334164890fcb6c900122ce` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:29.405339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7324b8c749334164890fcb6c900122ce)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:29.408195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:29.826219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7324b8c749334164890fcb6c900122ce'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.828169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79578cfa5efb4cb1ba9e926639641a55', 'request_id': 'req-db87ae9e-7ae3-4047-aaa1-78866d16a63a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.844784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79578cfa5efb4cb1ba9e926639641a55', 'request_id': 'req-db87ae9e-7ae3-4047-aaa1-78866d16a63a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.958916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7324b8c749334164890fcb6c900122ce'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.965926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7324b8c749334164890fcb6c900122ce'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.982887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7324b8c749334164890fcb6c900122ce'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:29.984010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a687828f-da2c-4d38-bdb2-a92791a2e2bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7324b8c749334164890fcb6c900122ce'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:30.000001 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1023/2050] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:29 2022] DELETE /identity/v3/users/7324b8c749334164890fcb6c900122ce => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:30.125916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79578cfa5efb4cb1ba9e926639641a55', 'request_id': 'req-db87ae9e-7ae3-4047-aaa1-78866d16a63a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:30.146550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] The token cache is being invalidate because project 79578cfa5efb4cb1ba9e926639641a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:30.147121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:30.148033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-db87ae9e-7ae3-4047-aaa1-78866d16a63a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:30.149878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1028/2051] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:29 2022] DELETE /identity/v3/projects/79578cfa5efb4cb1ba9e926639641a55 => generated 0 bytes in 965 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:30.160233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99a4e-d69f-4138-8b39-8167b844a3e1 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:30.160233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99a4e-d69f-4138-8b39-8167b844a3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:30.160233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99a4e-d69f-4138-8b39-8167b844a3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:30.190305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-00d99a4e-d69f-4138-8b39-8167b844a3e1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:30.192074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-00d99a4e-d69f-4138-8b39-8167b844a3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:30.307311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1024/2052] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:30.317249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca942f-a8bf-4a40-8099-6c1e4bcf7eea None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:30.321491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca942f-a8bf-4a40-8099-6c1e4bcf7eea None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:30.321491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ca942f-a8bf-4a40-8099-6c1e4bcf7eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:30.347620 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f1ca942f-a8bf-4a40-8099-6c1e4bcf7eea None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:30.349048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f1ca942f-a8bf-4a40-8099-6c1e4bcf7eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:30.365067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1029/2053] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:30.370194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d99a4e-d69f-4138-8b39-8167b844a3e1 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:30.382467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d99a4e-d69f-4138-8b39-8167b844a3e1 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:30.390638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcd279-bd35-4a98-a71b-503fb9c989a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:30.391798 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcd279-bd35-4a98-a71b-503fb9c989a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:30.397634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcd279-bd35-4a98-a71b-503fb9c989a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:30.397634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcd279-bd35-4a98-a71b-503fb9c989a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:30.397634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcd279-bd35-4a98-a71b-503fb9c989a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:30.397634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcd279-bd35-4a98-a71b-503fb9c989a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:30.400813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcd279-bd35-4a98-a71b-503fb9c989a9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:30.425598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1025/2054] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:30.429735 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca942f-a8bf-4a40-8099-6c1e4bcf7eea None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:30.455948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ca942f-a8bf-4a40-8099-6c1e4bcf7eea None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:30.482390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64976ba-b6e3-4471-948d-b5476440d19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:30.483958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64976ba-b6e3-4471-948d-b5476440d19f admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:30.484572 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64976ba-b6e3-4471-948d-b5476440d19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:30.486142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64976ba-b6e3-4471-948d-b5476440d19f admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:30.487745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64976ba-b6e3-4471-948d-b5476440d19f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:30.493407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64976ba-b6e3-4471-948d-b5476440d19f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:30.526848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:30.580846 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:30.622591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1030/2055] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:30 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:30.629027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcd279-bd35-4a98-a71b-503fb9c989a9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:30.649442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcd279-bd35-4a98-a71b-503fb9c989a9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:30.795598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaca5f-2a03-4a16-9f95-8b4e9be03797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:30.795598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaca5f-2a03-4a16-9f95-8b4e9be03797 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:30.795598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaca5f-2a03-4a16-9f95-8b4e9be03797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:30.796155 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaca5f-2a03-4a16-9f95-8b4e9be03797 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:30.798187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaca5f-2a03-4a16-9f95-8b4e9be03797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:30.800781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaca5f-2a03-4a16-9f95-8b4e9be03797 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:31.130965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64976ba-b6e3-4471-948d-b5476440d19f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:31.133889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f64976ba-b6e3-4471-948d-b5476440d19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:31.158312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64976ba-b6e3-4471-948d-b5476440d19f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:31.165281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:31.166027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:31.166417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:31.167685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] PATH_INFO: `/v3/projects/37b23fdc79934b9cb25959e8407dfcef` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:31.169771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37b23fdc79934b9cb25959e8407dfcef)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:31.173105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:31.187468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1026/2056] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:30 2022] POST /identity/v3/users => generated 288 bytes in 560 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:31.198214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaca5f-2a03-4a16-9f95-8b4e9be03797 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:31.225311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaca5f-2a03-4a16-9f95-8b4e9be03797 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:31.257358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb888f2-2a1b-4c3b-937d-124b6190241c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:31.260651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb888f2-2a1b-4c3b-937d-124b6190241c admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:31.260651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb888f2-2a1b-4c3b-937d-124b6190241c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:31.260651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb888f2-2a1b-4c3b-937d-124b6190241c admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:31.261298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb888f2-2a1b-4c3b-937d-124b6190241c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:31.267142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb888f2-2a1b-4c3b-937d-124b6190241c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:31.296292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1027/2057] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:31.303129 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb888f2-2a1b-4c3b-937d-124b6190241c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:31.321833 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb888f2-2a1b-4c3b-937d-124b6190241c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:31.331514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39fccb-9009-419e-a019-2a02aa0a1a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:31.332489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39fccb-9009-419e-a019-2a02aa0a1a24 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:31.333026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39fccb-9009-419e-a019-2a02aa0a1a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:31.333580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39fccb-9009-419e-a019-2a02aa0a1a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:31.335091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39fccb-9009-419e-a019-2a02aa0a1a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:31.340475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39fccb-9009-419e-a019-2a02aa0a1a24 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:31.363618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1028/2058] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:31.368881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39fccb-9009-419e-a019-2a02aa0a1a24 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:31.375088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37b23fdc79934b9cb25959e8407dfcef', 'request_id': 'req-d1d2f952-ac1c-4a59-be6b-7e13848996e6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:31.387278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39fccb-9009-419e-a019-2a02aa0a1a24 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:31.388263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37b23fdc79934b9cb25959e8407dfcef', 'request_id': 'req-d1d2f952-ac1c-4a59-be6b-7e13848996e6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:31.485728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f0a80a-04ba-4b18-8193-4862b96ec8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:31.486409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a80a-04ba-4b18-8193-4862b96ec8e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:31.486686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a80a-04ba-4b18-8193-4862b96ec8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:31.486961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f0a80a-04ba-4b18-8193-4862b96ec8e4 admin admin] PATH_INFO: `/v3/projects/6d4fa6545ff347e7b125d3921a8cc0dd/users/0e141315d9f7419686d355e7490bce3c/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:31.506513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0a80a-04ba-4b18-8193-4862b96ec8e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d4fa6545ff347e7b125d3921a8cc0dd, user_id=0e141315d9f7419686d355e7490bce3c, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:31.509476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f0a80a-04ba-4b18-8193-4862b96ec8e4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:31.530405 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37b23fdc79934b9cb25959e8407dfcef', 'request_id': 'req-d1d2f952-ac1c-4a59-be6b-7e13848996e6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:31.545822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] The token cache is being invalidate because project 37b23fdc79934b9cb25959e8407dfcef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:31.546320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:31.547460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:31.550108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1031/2059] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:31 2022] DELETE /identity/v3/projects/37b23fdc79934b9cb25959e8407dfcef => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:31.556550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:31.560050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:31.585251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d2f952-ac1c-4a59-be6b-7e13848996e6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:31.592923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:31.593812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:31.594097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:31.594486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] PATH_INFO: `/v3/users/6c3ae649482548b185322ba02f298521` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:31.596743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c3ae649482548b185322ba02f298521)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:31.600695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:31.630327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1029/2060] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:31 2022] PUT /identity/v3/projects/6d4fa6545ff347e7b125d3921a8cc0dd/users/0e141315d9f7419686d355e7490bce3c/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:31.649059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ee7a5-7cf4-4819-a028-4b9ce9d7f1f1 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:31.649059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ee7a5-7cf4-4819-a028-4b9ce9d7f1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:31.649059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ee7a5-7cf4-4819-a028-4b9ce9d7f1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:31.686933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e05ee7a5-7cf4-4819-a028-4b9ce9d7f1f1 None None] MFA Rules not processed for user `0e141315d9f7419686d355e7490bce3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:31.691353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e05ee7a5-7cf4-4819-a028-4b9ce9d7f1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:31.805430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1030/2061] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:31.850374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6410048-df0c-4ed9-bdde-b4a1d4d01752 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:31.850374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6410048-df0c-4ed9-bdde-b4a1d4d01752 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:31.850374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6410048-df0c-4ed9-bdde-b4a1d4d01752 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:31.850374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1031/2062] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:31:31 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:31:31.850374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6410048-df0c-4ed9-bdde-b4a1d4d01752 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:31.850374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a6410048-df0c-4ed9-bdde-b4a1d4d01752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:31.967399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c3ae649482548b185322ba02f298521'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:31.969059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c3ae649482548b185322ba02f298521'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:31.969253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6410048-df0c-4ed9-bdde-b4a1d4d01752 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:31.970360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c3ae649482548b185322ba02f298521'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:31.976420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fec0a-1994-4cae-97c0-395926528b39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:31.977489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fec0a-1994-4cae-97c0-395926528b39 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:31.977989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fec0a-1994-4cae-97c0-395926528b39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:31.978767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fec0a-1994-4cae-97c0-395926528b39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:31.983033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c03fec0a-1994-4cae-97c0-395926528b39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:31.986922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c3ae649482548b185322ba02f298521'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:32.006916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fec0a-1994-4cae-97c0-395926528b39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:32.008724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03fec0a-1994-4cae-97c0-395926528b39 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:32.015181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c3ae649482548b185322ba02f298521'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:32.026502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1032/2063] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:32.030653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80e8fd-f801-49e3-9f2e-0730afdf7a64 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:32.038829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80e8fd-f801-49e3-9f2e-0730afdf7a64 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:32.038829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80e8fd-f801-49e3-9f2e-0730afdf7a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:32.078008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-3c80e8fd-f801-49e3-9f2e-0730afdf7a64 None None] MFA Rules not processed for user `9f73bca3896440258430cfce042faa42`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:32.080532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3c80e8fd-f801-49e3-9f2e-0730afdf7a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:32.151085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1032/2064] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:31 2022] DELETE /identity/v3/users/6c3ae649482548b185322ba02f298521 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:32.153441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:32.160829 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:32.249309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1033/2065] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:31:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:32.259297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:32.275193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735cece-2ad0-4984-bba0-1b110dddd030 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:32.275593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735cece-2ad0-4984-bba0-1b110dddd030 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:32.275868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735cece-2ad0-4984-bba0-1b110dddd030 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:32.277851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1034/2066] 10.222.0.64 () {54 vars in 961 bytes} [Fri Sep 23 01:31:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:31:32.287501 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b85f5b-d2c9-4bf1-b54d-c568760fa0c7 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:32.288048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b85f5b-d2c9-4bf1-b54d-c568760fa0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:32.288512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b85f5b-d2c9-4bf1-b54d-c568760fa0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:32.325718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49304b06-ca37-4ae5-97a7-b76648341e8f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:32.337104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-70b85f5b-d2c9-4bf1-b54d-c568760fa0c7 None None] MFA Rules not processed for user `9f73bca3896440258430cfce042faa42`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:32.338650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:32.339783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-70b85f5b-d2c9-4bf1-b54d-c568760fa0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:32.340785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:32.342391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:32.343118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] PATH_INFO: `/v3/users/1f628503bfd24338b4a351d05a18f126` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:32.347665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f628503bfd24338b4a351d05a18f126)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:32.354409 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:32.482038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1035/2067] 10.222.0.64 () {58 vars in 1058 bytes} [Fri Sep 23 01:31:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:32.495859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b85f5b-d2c9-4bf1-b54d-c568760fa0c7 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:32.514756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b85f5b-d2c9-4bf1-b54d-c568760fa0c7 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:32.525497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb17d34-aea0-4645-a92f-f3d68ee93070 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:32.526677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb17d34-aea0-4645-a92f-f3d68ee93070 service placement] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:32.527177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb17d34-aea0-4645-a92f-f3d68ee93070 service placement] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:32.528486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb17d34-aea0-4645-a92f-f3d68ee93070 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:32.546785 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb17d34-aea0-4645-a92f-f3d68ee93070 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:32.556759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb17d34-aea0-4645-a92f-f3d68ee93070 service placement] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:32.575894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1036/2068] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:32.645268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f628503bfd24338b4a351d05a18f126'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:32.647347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f628503bfd24338b4a351d05a18f126'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:32.648622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f628503bfd24338b4a351d05a18f126'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:32.661785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f628503bfd24338b4a351d05a18f126'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:32.678363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f628503bfd24338b4a351d05a18f126'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:32.771864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1033/2069] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:32 2022] DELETE /identity/v3/users/1f628503bfd24338b4a351d05a18f126 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:32.899227 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:33.181265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb17d34-aea0-4645-a92f-f3d68ee93070 service placement] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:33.183455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-feb17d34-aea0-4645-a92f-f3d68ee93070 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:33.193905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:33.196295 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:33.261564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb17d34-aea0-4645-a92f-f3d68ee93070 service placement] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:33.359878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:33.359878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:33.359878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:33.359878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] PATH_INFO: `/v3/users/8eb38c1cf4c14eed9d51e0b649cbf993` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:33.360468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eb38c1cf4c14eed9d51e0b649cbf993)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:33.367281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:33.367882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5b269-0fca-45d4-bab0-9ebe0259dda4 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:33.386981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:33.388115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:33.388655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:33.389157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] PATH_INFO: `/v3/projects/4796b1219050424888e9e474a7df1b66` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:33.392973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4796b1219050424888e9e474a7df1b66)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:33.396148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:33.686629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4796b1219050424888e9e474a7df1b66', 'request_id': 'req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:33.691878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eb38c1cf4c14eed9d51e0b649cbf993'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:33.696308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4796b1219050424888e9e474a7df1b66', 'request_id': 'req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:33.821830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eb38c1cf4c14eed9d51e0b649cbf993'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:33.823325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eb38c1cf4c14eed9d51e0b649cbf993'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:33.847845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eb38c1cf4c14eed9d51e0b649cbf993'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:33.849978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eb38c1cf4c14eed9d51e0b649cbf993'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:33.852307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:33.878466 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1037/2070] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:33 2022] DELETE /identity/v3/users/8eb38c1cf4c14eed9d51e0b649cbf993 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:33.880269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:33.882803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:33.884144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4796b1219050424888e9e474a7df1b66', 'request_id': 'req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:33.898232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] The token cache is being invalidate because project 4796b1219050424888e9e474a7df1b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:33.898653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:33.899466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2bbfffe7-e60b-4c82-9a81-fb4b1e19c6a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:33.901107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1034/2071] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:33 2022] DELETE /identity/v3/projects/4796b1219050424888e9e474a7df1b66 => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:33.902666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc73328-e0fe-4f7c-bcc0-2931a5ddb5aa None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:33.903221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc73328-e0fe-4f7c-bcc0-2931a5ddb5aa None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:33.903649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc73328-e0fe-4f7c-bcc0-2931a5ddb5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:33.929421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6fc73328-e0fe-4f7c-bcc0-2931a5ddb5aa None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:33.930751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6fc73328-e0fe-4f7c-bcc0-2931a5ddb5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:33.985406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1819c-506a-4be9-b55d-2149cc0164ae admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:33.991981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:33.992765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:33.993143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:33.993525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] PATH_INFO: `/v3/projects/a90db67f8f74446b8cbd913a2a17c0b1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:33.994877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=a90db67f8f74446b8cbd913a2a17c0b1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:33.997621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:34.025492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1035/2072] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:34.027075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93767f06-e3bf-4b89-8192-89e930ebe65f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.027367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93767f06-e3bf-4b89-8192-89e930ebe65f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.027726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93767f06-e3bf-4b89-8192-89e930ebe65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.059788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-93767f06-e3bf-4b89-8192-89e930ebe65f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:34.061250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-93767f06-e3bf-4b89-8192-89e930ebe65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:34.069418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1036/2073] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:34.070275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93767f06-e3bf-4b89-8192-89e930ebe65f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:34.071644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-93767f06-e3bf-4b89-8192-89e930ebe65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:34.138310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93767f06-e3bf-4b89-8192-89e930ebe65f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:34.142974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabccda9-aa08-456d-b08b-257e8b873e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:34.143801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabccda9-aa08-456d-b08b-257e8b873e87 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.144096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabccda9-aa08-456d-b08b-257e8b873e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.144402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabccda9-aa08-456d-b08b-257e8b873e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.158799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabccda9-aa08-456d-b08b-257e8b873e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:34.160319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabccda9-aa08-456d-b08b-257e8b873e87 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:34.161217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a90db67f8f74446b8cbd913a2a17c0b1', 'request_id': 'req-cd50ef51-e424-4220-a1fe-af0f55a856ca'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:34.172319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1037/2074] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:34.173320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a90db67f8f74446b8cbd913a2a17c0b1', 'request_id': 'req-cd50ef51-e424-4220-a1fe-af0f55a856ca'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:34.173903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08000c-e459-4008-a66f-605c17fcf054 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.174381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08000c-e459-4008-a66f-605c17fcf054 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.174655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08000c-e459-4008-a66f-605c17fcf054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.209759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-7a08000c-e459-4008-a66f-605c17fcf054 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:34.212280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7a08000c-e459-4008-a66f-605c17fcf054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:34.314048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a90db67f8f74446b8cbd913a2a17c0b1', 'request_id': 'req-cd50ef51-e424-4220-a1fe-af0f55a856ca'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:34.320495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1038/2075] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:34.327471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08000c-e459-4008-a66f-605c17fcf054 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:34.337888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08000c-e459-4008-a66f-605c17fcf054 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:34.346628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7cbe9-1b04-4374-95ec-bd39eeada0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:34.347375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7cbe9-1b04-4374-95ec-bd39eeada0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.347823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7cbe9-1b04-4374-95ec-bd39eeada0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.348246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7cbe9-1b04-4374-95ec-bd39eeada0d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.348848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7cbe9-1b04-4374-95ec-bd39eeada0d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:34.349164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7cbe9-1b04-4374-95ec-bd39eeada0d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:34.350862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7cbe9-1b04-4374-95ec-bd39eeada0d5 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:34.357546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] The token cache is being invalidate because project a90db67f8f74446b8cbd913a2a17c0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:34.358061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:34.359216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd50ef51-e424-4220-a1fe-af0f55a856ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:34.362199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1038/2076] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:33 2022] DELETE /identity/v3/projects/a90db67f8f74446b8cbd913a2a17c0b1 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:34.377949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1039/2077] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:34.379121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec881e9c-189d-450b-a868-db3a287b3048 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.379751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec881e9c-189d-450b-a868-db3a287b3048 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.380283 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec881e9c-189d-450b-a868-db3a287b3048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.384987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7cbe9-1b04-4374-95ec-bd39eeada0d5 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:34.387811 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-25a7cbe9-1b04-4374-95ec-bd39eeada0d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:34.417543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7cbe9-1b04-4374-95ec-bd39eeada0d5 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:34.425552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c38c9a-faa0-4ec6-8020-de07663d753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:34.426523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c38c9a-faa0-4ec6-8020-de07663d753d admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.426931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c38c9a-faa0-4ec6-8020-de07663d753d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.427365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c38c9a-faa0-4ec6-8020-de07663d753d admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.428233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c38c9a-faa0-4ec6-8020-de07663d753d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:34.435500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c38c9a-faa0-4ec6-8020-de07663d753d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:34.435950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ec881e9c-189d-450b-a868-db3a287b3048 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:34.437354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ec881e9c-189d-450b-a868-db3a287b3048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:34.448338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1039/2078] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:34.472347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915736b-702b-4691-8e90-04b757c16e37 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.472347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915736b-702b-4691-8e90-04b757c16e37 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.472347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915736b-702b-4691-8e90-04b757c16e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.498714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e915736b-702b-4691-8e90-04b757c16e37 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:34.500905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e915736b-702b-4691-8e90-04b757c16e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:34.514999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1040/2079] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:34.527780 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915736b-702b-4691-8e90-04b757c16e37 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:34.542785 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915736b-702b-4691-8e90-04b757c16e37 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:34.564251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6f95-0c40-406d-ad1a-d984ece2a6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:34.564251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6f95-0c40-406d-ad1a-d984ece2a6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.564251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6f95-0c40-406d-ad1a-d984ece2a6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.564251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6f95-0c40-406d-ad1a-d984ece2a6d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.567800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6f95-0c40-406d-ad1a-d984ece2a6d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:34.567800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6f95-0c40-406d-ad1a-d984ece2a6d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:34.567800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6f95-0c40-406d-ad1a-d984ece2a6d5 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:34.600691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1041/2080] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:31:34.608646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6f95-0c40-406d-ad1a-d984ece2a6d5 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:34.614637 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1040/2081] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:34 2022] POST /identity/v3/projects => generated 385 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:34.620883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c38c9a-faa0-4ec6-8020-de07663d753d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:34.624936 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6f95-0c40-406d-ad1a-d984ece2a6d5 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:34.712542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a0afe-c9ff-495b-8b86-e26ef25ee6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:34.713194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a0afe-c9ff-495b-8b86-e26ef25ee6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.713454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a0afe-c9ff-495b-8b86-e26ef25ee6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.713856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a0afe-c9ff-495b-8b86-e26ef25ee6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.714556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a0afe-c9ff-495b-8b86-e26ef25ee6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:34.718482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a0afe-c9ff-495b-8b86-e26ef25ee6d1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:34.764931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c38c9a-faa0-4ec6-8020-de07663d753d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:34.770911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2217579-c2fb-43eb-a66b-39791e576d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:34.771583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2217579-c2fb-43eb-a66b-39791e576d7a admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.771850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2217579-c2fb-43eb-a66b-39791e576d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.772126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2217579-c2fb-43eb-a66b-39791e576d7a admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.772782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2217579-c2fb-43eb-a66b-39791e576d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:34.775266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2217579-c2fb-43eb-a66b-39791e576d7a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:34.872406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1042/2082] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:34 2022] POST /identity/v3/projects => generated 383 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:34.880209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a0afe-c9ff-495b-8b86-e26ef25ee6d1 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:34.901693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1041/2083] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:34 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:31:34.906669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2217579-c2fb-43eb-a66b-39791e576d7a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:34.919346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2217579-c2fb-43eb-a66b-39791e576d7a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:34.974796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a0afe-c9ff-495b-8b86-e26ef25ee6d1 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:34.980727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d53c4-3622-4d64-8395-27e3da534ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:34.981502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d53c4-3622-4d64-8395-27e3da534ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:34.981895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d53c4-3622-4d64-8395-27e3da534ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:34.982869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d53c4-3622-4d64-8395-27e3da534ba7 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:34.983889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d53c4-3622-4d64-8395-27e3da534ba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:34.986561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d53c4-3622-4d64-8395-27e3da534ba7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:35.016251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fa982-43dd-4f60-b722-7b7289a90fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:35.017237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fa982-43dd-4f60-b722-7b7289a90fec admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.017684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fa982-43dd-4f60-b722-7b7289a90fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.018192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fa982-43dd-4f60-b722-7b7289a90fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:35.019140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fa982-43dd-4f60-b722-7b7289a90fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:35.021370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fa982-43dd-4f60-b722-7b7289a90fec admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:35.033565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1042/2084] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:35.034450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fa982-43dd-4f60-b722-7b7289a90fec admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:35.036341 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-519fa982-43dd-4f60-b722-7b7289a90fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:35.049445 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fa982-43dd-4f60-b722-7b7289a90fec admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:35.055012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:35.055820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.056098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.056372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] PATH_INFO: `/v3/projects/0ae858c183c04be58a7fc3ccfd0b8649` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:35.057595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae858c183c04be58a7fc3ccfd0b8649)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:35.059624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:35.102892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:35.111268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1043/2085] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:34 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:31:35.112108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d53c4-3622-4d64-8395-27e3da534ba7 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:35.124551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d53c4-3622-4d64-8395-27e3da534ba7 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:35.132361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bebaf3-8ca2-429d-a924-c2d9b9441843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:35.133264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bebaf3-8ca2-429d-a924-c2d9b9441843 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.133530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bebaf3-8ca2-429d-a924-c2d9b9441843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.133844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bebaf3-8ca2-429d-a924-c2d9b9441843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:35.134568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bebaf3-8ca2-429d-a924-c2d9b9441843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:35.136530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bebaf3-8ca2-429d-a924-c2d9b9441843 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:35.149670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1044/2086] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:35.150737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bebaf3-8ca2-429d-a924-c2d9b9441843 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:35.162057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bebaf3-8ca2-429d-a924-c2d9b9441843 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:35.167243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3ab79-c69b-45ff-9f53-b82dbada6ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:35.167954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3ab79-c69b-45ff-9f53-b82dbada6ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.168341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3ab79-c69b-45ff-9f53-b82dbada6ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.168562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3ab79-c69b-45ff-9f53-b82dbada6ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:35.169158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3ab79-c69b-45ff-9f53-b82dbada6ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:35.171351 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3ab79-c69b-45ff-9f53-b82dbada6ae8 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:35.186609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1045/2087] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:35.191675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3ab79-c69b-45ff-9f53-b82dbada6ae8 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:35.202216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3ab79-c69b-45ff-9f53-b82dbada6ae8 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:35.210734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5ba89-dd38-4a7d-b479-0fde61069d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:35.211731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5ba89-dd38-4a7d-b479-0fde61069d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.212271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5ba89-dd38-4a7d-b479-0fde61069d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.212555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5ba89-dd38-4a7d-b479-0fde61069d94 admin admin] PATH_INFO: `/v3/projects/4a56af79aca1481ea87944a95624b3d1/users/fca7412ff12d4fcdbbcf996f67213864/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:35.231324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5ba89-dd38-4a7d-b479-0fde61069d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a56af79aca1481ea87944a95624b3d1, user_id=fca7412ff12d4fcdbbcf996f67213864, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:35.235439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5ba89-dd38-4a7d-b479-0fde61069d94 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:35.245890 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae858c183c04be58a7fc3ccfd0b8649', 'request_id': 'req-6323240a-de4c-4a96-bbdf-780b81fa7c71'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:35.254142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae858c183c04be58a7fc3ccfd0b8649', 'request_id': 'req-6323240a-de4c-4a96-bbdf-780b81fa7c71'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:35.331705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1046/2088] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:35 2022] PUT /identity/v3/projects/4a56af79aca1481ea87944a95624b3d1/users/fca7412ff12d4fcdbbcf996f67213864/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:35.345742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5ba89-dd38-4a7d-b479-0fde61069d94 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:35.359041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae858c183c04be58a7fc3ccfd0b8649', 'request_id': 'req-6323240a-de4c-4a96-bbdf-780b81fa7c71'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:35.375234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] The token cache is being invalidate because project 0ae858c183c04be58a7fc3ccfd0b8649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:35.375542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:35.376292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6323240a-de4c-4a96-bbdf-780b81fa7c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:35.377919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1043/2089] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:35 2022] DELETE /identity/v3/projects/0ae858c183c04be58a7fc3ccfd0b8649 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:35.379202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d2ac-4cdb-40c9-ad9f-93bd2795c3fa None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.379863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d2ac-4cdb-40c9-ad9f-93bd2795c3fa None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.380144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4d2ac-4cdb-40c9-ad9f-93bd2795c3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:35.406839 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5ba89-dd38-4a7d-b479-0fde61069d94 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:35.411733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-dcf4d2ac-4cdb-40c9-ad9f-93bd2795c3fa None None] MFA Rules not processed for user `fca7412ff12d4fcdbbcf996f67213864`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:35.411972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8ab8e-08bc-4a14-83e9-a8d1a65019a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:35.412603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8ab8e-08bc-4a14-83e9-a8d1a65019a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.412772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-dcf4d2ac-4cdb-40c9-ad9f-93bd2795c3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:35.412874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8ab8e-08bc-4a14-83e9-a8d1a65019a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.413135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8ab8e-08bc-4a14-83e9-a8d1a65019a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:35.413802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8ab8e-08bc-4a14-83e9-a8d1a65019a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:35.415408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8ab8e-08bc-4a14-83e9-a8d1a65019a7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:35.426590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1047/2090] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:35.427476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8ab8e-08bc-4a14-83e9-a8d1a65019a7 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:35.429345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-adc8ab8e-08bc-4a14-83e9-a8d1a65019a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:35.452496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8ab8e-08bc-4a14-83e9-a8d1a65019a7 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:35.463702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:35.464326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.464609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.465616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] PATH_INFO: `/v3/users/8313f22a39424320870259c32c2035cb` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:35.467821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8313f22a39424320870259c32c2035cb)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:35.471363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:35.505608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1044/2091] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:35.506568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d2ac-4cdb-40c9-ad9f-93bd2795c3fa None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:35.508487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-dcf4d2ac-4cdb-40c9-ad9f-93bd2795c3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:35.529333 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4d2ac-4cdb-40c9-ad9f-93bd2795c3fa None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:35.535054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a399d-e679-4dc0-96b7-0e8edb9edf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:35.535948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a399d-e679-4dc0-96b7-0e8edb9edf63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.536354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a399d-e679-4dc0-96b7-0e8edb9edf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.536773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a399d-e679-4dc0-96b7-0e8edb9edf63 admin admin] PATH_INFO: `/v3/projects/9b86db5b3dc3461a803e8541509ed2fb/users/43dbde949334471fbf157054a070bcfe/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:35.559741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a399d-e679-4dc0-96b7-0e8edb9edf63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b86db5b3dc3461a803e8541509ed2fb, user_id=43dbde949334471fbf157054a070bcfe, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:35.567678 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a399d-e679-4dc0-96b7-0e8edb9edf63 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:35.705426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1045/2092] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:35 2022] PUT /identity/v3/projects/9b86db5b3dc3461a803e8541509ed2fb/users/43dbde949334471fbf157054a070bcfe/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:35.705426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a399d-e679-4dc0-96b7-0e8edb9edf63 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:35.717893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-860a399d-e679-4dc0-96b7-0e8edb9edf63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:35.790608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8313f22a39424320870259c32c2035cb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:35.873766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8313f22a39424320870259c32c2035cb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:35.874553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8313f22a39424320870259c32c2035cb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:35.882483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a399d-e679-4dc0-96b7-0e8edb9edf63 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:35.894177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b5cc0-6ba2-41d0-ba53-f3572ccb8153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:35.895068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b5cc0-6ba2-41d0-ba53-f3572ccb8153 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.895768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b5cc0-6ba2-41d0-ba53-f3572ccb8153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.896187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b5cc0-6ba2-41d0-ba53-f3572ccb8153 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:35.897068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b5cc0-6ba2-41d0-ba53-f3572ccb8153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:35.900378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b5cc0-6ba2-41d0-ba53-f3572ccb8153 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:35.938354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8313f22a39424320870259c32c2035cb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:35.940760 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cd3875b9-9ce3-42d8-a6f9-4575dc14a4a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8313f22a39424320870259c32c2035cb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:35.964853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1048/2093] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:35 2022] DELETE /identity/v3/users/8313f22a39424320870259c32c2035cb => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:35.967367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac50d6d-9df1-462f-8fff-3b862f06906f None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:35.967367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac50d6d-9df1-462f-8fff-3b862f06906f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:35.967725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac50d6d-9df1-462f-8fff-3b862f06906f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:36.008418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8ac50d6d-9df1-462f-8fff-3b862f06906f None None] MFA Rules not processed for user `43dbde949334471fbf157054a070bcfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:36.010117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8ac50d6d-9df1-462f-8fff-3b862f06906f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:36.096782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1046/2094] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:35 2022] POST /identity/v3/projects => generated 385 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:36.103019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b5cc0-6ba2-41d0-ba53-f3572ccb8153 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:36.106646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-084b5cc0-6ba2-41d0-ba53-f3572ccb8153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:36.123654 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1049/2095] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:36.128489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac50d6d-9df1-462f-8fff-3b862f06906f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:36.129909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8ac50d6d-9df1-462f-8fff-3b862f06906f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:36.220296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac50d6d-9df1-462f-8fff-3b862f06906f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:36.220819 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b5cc0-6ba2-41d0-ba53-f3572ccb8153 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:36.227041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57758a2-29a6-4a6c-b5dd-e46941f841f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:36.228128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57758a2-29a6-4a6c-b5dd-e46941f841f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:36.228802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57758a2-29a6-4a6c-b5dd-e46941f841f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:36.229373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57758a2-29a6-4a6c-b5dd-e46941f841f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:36.230221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57758a2-29a6-4a6c-b5dd-e46941f841f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:36.231176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2e745-e2f8-4f4f-a454-63b54795582d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:36.233128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2e745-e2f8-4f4f-a454-63b54795582d admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:36.233849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2e745-e2f8-4f4f-a454-63b54795582d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:36.234153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57758a2-29a6-4a6c-b5dd-e46941f841f1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:36.234522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2e745-e2f8-4f4f-a454-63b54795582d admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:36.235862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2e745-e2f8-4f4f-a454-63b54795582d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:36.240198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2e745-e2f8-4f4f-a454-63b54795582d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:36.249835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:36.655829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1050/2096] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:36 2022] POST /identity/v3/projects => generated 381 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:31:36.657394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57758a2-29a6-4a6c-b5dd-e46941f841f1 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:36.660040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1047/2097] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:36 2022] POST /identity/v3/users => generated 293 bytes in 559 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:36.664182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2e745-e2f8-4f4f-a454-63b54795582d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:36.675239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2e745-e2f8-4f4f-a454-63b54795582d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:36.726952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57758a2-29a6-4a6c-b5dd-e46941f841f1 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:36.745376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c7926-7879-4b6e-87bf-521e6db23358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:36.745376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c7926-7879-4b6e-87bf-521e6db23358 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:36.745376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c7926-7879-4b6e-87bf-521e6db23358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:36.745376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c7926-7879-4b6e-87bf-521e6db23358 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:36.745861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c7926-7879-4b6e-87bf-521e6db23358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:36.745861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c7926-7879-4b6e-87bf-521e6db23358 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:36.757241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161aa9af-8527-41f7-925c-3b823574e399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:36.758298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161aa9af-8527-41f7-925c-3b823574e399 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:36.758782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161aa9af-8527-41f7-925c-3b823574e399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:36.759228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161aa9af-8527-41f7-925c-3b823574e399 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:36.760467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161aa9af-8527-41f7-925c-3b823574e399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:36.764502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161aa9af-8527-41f7-925c-3b823574e399 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:36.785480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1048/2098] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:36.789699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161aa9af-8527-41f7-925c-3b823574e399 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:36.800985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161aa9af-8527-41f7-925c-3b823574e399 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:36.806117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0871e1-09a8-4577-85ff-3f804745eaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:36.806858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0871e1-09a8-4577-85ff-3f804745eaa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:36.807175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0871e1-09a8-4577-85ff-3f804745eaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:36.807518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0871e1-09a8-4577-85ff-3f804745eaa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:36.808274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0871e1-09a8-4577-85ff-3f804745eaa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:36.810110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0871e1-09a8-4577-85ff-3f804745eaa8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:36.830098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1049/2099] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:36.840991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0871e1-09a8-4577-85ff-3f804745eaa8 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:36.853609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0871e1-09a8-4577-85ff-3f804745eaa8 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:36.861460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcacd90-f6d4-413d-836c-6fcecdbcc174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:36.862374 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcacd90-f6d4-413d-836c-6fcecdbcc174 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:36.862781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcacd90-f6d4-413d-836c-6fcecdbcc174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:36.863100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcacd90-f6d4-413d-836c-6fcecdbcc174 admin admin] PATH_INFO: `/v3/projects/cf2cdcc5b125423c8a3dedf5a4cf9e32/users/7532a489d0b74122a5ff3069fd9997a0/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:36.882728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcacd90-f6d4-413d-836c-6fcecdbcc174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf2cdcc5b125423c8a3dedf5a4cf9e32, user_id=7532a489d0b74122a5ff3069fd9997a0, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:36.890728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcacd90-f6d4-413d-836c-6fcecdbcc174 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:36.934260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1051/2100] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:36 2022] POST /identity/v3/users => generated 297 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:36.934260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c7926-7879-4b6e-87bf-521e6db23358 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:36.937402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a42c7926-7879-4b6e-87bf-521e6db23358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:36.959214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c7926-7879-4b6e-87bf-521e6db23358 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:36.967051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:36.968234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:36.968749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:36.969257 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] PATH_INFO: `/v3/projects/614af2c2a5c8452cb9e70eb607cb5d4a` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:36.971354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=614af2c2a5c8452cb9e70eb607cb5d4a)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:36.975099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:37.069750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1050/2101] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:36 2022] PUT /identity/v3/projects/cf2cdcc5b125423c8a3dedf5a4cf9e32/users/7532a489d0b74122a5ff3069fd9997a0/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:37.081644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcacd90-f6d4-413d-836c-6fcecdbcc174 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:37.193464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614af2c2a5c8452cb9e70eb607cb5d4a', 'request_id': 'req-fb634c97-489f-4661-8f39-e1d02ed0c8e6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:37.206328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614af2c2a5c8452cb9e70eb607cb5d4a', 'request_id': 'req-fb634c97-489f-4661-8f39-e1d02ed0c8e6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:37.209360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcacd90-f6d4-413d-836c-6fcecdbcc174 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:37.288275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae043f-1854-4f86-a724-fa3225c8eada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:37.288959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae043f-1854-4f86-a724-fa3225c8eada admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:37.289378 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae043f-1854-4f86-a724-fa3225c8eada admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:37.289544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae043f-1854-4f86-a724-fa3225c8eada admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:37.290228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae043f-1854-4f86-a724-fa3225c8eada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:37.291991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae043f-1854-4f86-a724-fa3225c8eada admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:37.305646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1051/2102] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:37.307196 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486d4f1-4a11-4443-8a45-b6b37506605c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:37.307721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486d4f1-4a11-4443-8a45-b6b37506605c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:37.308019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486d4f1-4a11-4443-8a45-b6b37506605c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:37.324482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614af2c2a5c8452cb9e70eb607cb5d4a', 'request_id': 'req-fb634c97-489f-4661-8f39-e1d02ed0c8e6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:37.342206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] The token cache is being invalidate because project 614af2c2a5c8452cb9e70eb607cb5d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:37.342600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:37.342916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9486d4f1-4a11-4443-8a45-b6b37506605c None None] MFA Rules not processed for user `7532a489d0b74122a5ff3069fd9997a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:37.344594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:37.344987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9486d4f1-4a11-4443-8a45-b6b37506605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:37.346038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1052/2103] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:36 2022] DELETE /identity/v3/projects/614af2c2a5c8452cb9e70eb607cb5d4a => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:37.346906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:37.348406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:37.364586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb634c97-489f-4661-8f39-e1d02ed0c8e6 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:37.370081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8c68f-6ab9-4335-b114-2f6f401b28a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:37.370858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8c68f-6ab9-4335-b114-2f6f401b28a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:37.371148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8c68f-6ab9-4335-b114-2f6f401b28a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:37.376515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8c68f-6ab9-4335-b114-2f6f401b28a4 admin admin] PATH_INFO: `/v3/projects/c26e334cbb7f48c5834f8acc68313043/users/f27281d4d3eb4933873a1547f69110cc/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:37.394204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8c68f-6ab9-4335-b114-2f6f401b28a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26e334cbb7f48c5834f8acc68313043, user_id=f27281d4d3eb4933873a1547f69110cc, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:37.398578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8c68f-6ab9-4335-b114-2f6f401b28a4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:37.437903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1052/2104] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:37.438957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486d4f1-4a11-4443-8a45-b6b37506605c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:37.440579 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9486d4f1-4a11-4443-8a45-b6b37506605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:37.456827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486d4f1-4a11-4443-8a45-b6b37506605c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:37.462256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:37.462966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:37.479271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:37.479271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] PATH_INFO: `/v3/users/7d605fa3f15e48e89ec249dfb6c4863b` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:37.479271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d605fa3f15e48e89ec249dfb6c4863b)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:37.479271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:37.541896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1053/2105] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:37 2022] PUT /identity/v3/projects/c26e334cbb7f48c5834f8acc68313043/users/f27281d4d3eb4933873a1547f69110cc/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:37.542747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8c68f-6ab9-4335-b114-2f6f401b28a4 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:37.544588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a7a8c68f-6ab9-4335-b114-2f6f401b28a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:37.628605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8c68f-6ab9-4335-b114-2f6f401b28a4 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:37.637023 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e0c3-87b9-450c-b948-93a898792044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:37.637982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e0c3-87b9-450c-b948-93a898792044 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:37.651040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e0c3-87b9-450c-b948-93a898792044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:37.651040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0e0c3-87b9-450c-b948-93a898792044 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:37.651040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0e0c3-87b9-450c-b948-93a898792044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:37.651040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0e0c3-87b9-450c-b948-93a898792044 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:37.796028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1054/2106] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:37 2022] POST /identity/v3/projects => generated 383 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:37.796028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e0c3-87b9-450c-b948-93a898792044 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:37.804334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d605fa3f15e48e89ec249dfb6c4863b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:37.805083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d605fa3f15e48e89ec249dfb6c4863b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:37.805816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d605fa3f15e48e89ec249dfb6c4863b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:37.818975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d605fa3f15e48e89ec249dfb6c4863b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:37.834869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d605fa3f15e48e89ec249dfb6c4863b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:37.899528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0e0c3-87b9-450c-b948-93a898792044 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:37.905604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d828a-f0f6-4535-ac9a-b10c7254ac34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:37.906243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d828a-f0f6-4535-ac9a-b10c7254ac34 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:37.906532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d828a-f0f6-4535-ac9a-b10c7254ac34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:37.906884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d828a-f0f6-4535-ac9a-b10c7254ac34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:37.907579 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d828a-f0f6-4535-ac9a-b10c7254ac34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:37.909355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d828a-f0f6-4535-ac9a-b10c7254ac34 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:37.921100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1055/2107] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:37.925218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d828a-f0f6-4535-ac9a-b10c7254ac34 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:37.926757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-289d828a-f0f6-4535-ac9a-b10c7254ac34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:37.942969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d828a-f0f6-4535-ac9a-b10c7254ac34 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:37.952099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:37.952872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1053/2108] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:37 2022] DELETE /identity/v3/users/7d605fa3f15e48e89ec249dfb6c4863b => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:37.953999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:37.956684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:37.974484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505237a1-2097-4ca3-a86f-2ebebb25b6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:37.975229 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505237a1-2097-4ca3-a86f-2ebebb25b6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:37.976150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505237a1-2097-4ca3-a86f-2ebebb25b6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:37.976444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505237a1-2097-4ca3-a86f-2ebebb25b6f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:37.977140 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505237a1-2097-4ca3-a86f-2ebebb25b6f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:37.980031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505237a1-2097-4ca3-a86f-2ebebb25b6f2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:38.069606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adbefe2-3094-4477-8b4a-c16688b327ed admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:38.075307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2061ec0-9b39-4a4e-a2af-fd119c3fbbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:38.076239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2061ec0-9b39-4a4e-a2af-fd119c3fbbe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:38.076463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2061ec0-9b39-4a4e-a2af-fd119c3fbbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:38.076806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2061ec0-9b39-4a4e-a2af-fd119c3fbbe1 admin admin] PATH_INFO: `/v3/projects/c26e334cbb7f48c5834f8acc68313043/users/f27281d4d3eb4933873a1547f69110cc/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:38.078973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2061ec0-9b39-4a4e-a2af-fd119c3fbbe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26e334cbb7f48c5834f8acc68313043, user_id=f27281d4d3eb4933873a1547f69110cc, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:38.081822 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2061ec0-9b39-4a4e-a2af-fd119c3fbbe1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:38.095317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1054/2109] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:37 2022] PUT /identity/v3/projects/c26e334cbb7f48c5834f8acc68313043/users/f27281d4d3eb4933873a1547f69110cc/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:38.109320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03305def-f2c3-4147-bb0c-3ad38d05ab09 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:38.109818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03305def-f2c3-4147-bb0c-3ad38d05ab09 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:38.110295 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03305def-f2c3-4147-bb0c-3ad38d05ab09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:38.147599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1056/2110] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:37 2022] POST /identity/v3/users => generated 298 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:38.152504 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505237a1-2097-4ca3-a86f-2ebebb25b6f2 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:38.153089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-03305def-f2c3-4147-bb0c-3ad38d05ab09 None None] MFA Rules not processed for user `f27281d4d3eb4933873a1547f69110cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:38.156444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-03305def-f2c3-4147-bb0c-3ad38d05ab09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:38.164423 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505237a1-2097-4ca3-a86f-2ebebb25b6f2 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:38.243029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c108310-4a97-4ea0-ac55-b13c378cc59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:38.243941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c108310-4a97-4ea0-ac55-b13c378cc59f admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:38.244357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c108310-4a97-4ea0-ac55-b13c378cc59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:38.244723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c108310-4a97-4ea0-ac55-b13c378cc59f admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:38.245547 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c108310-4a97-4ea0-ac55-b13c378cc59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:38.247723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c108310-4a97-4ea0-ac55-b13c378cc59f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:38.260910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1057/2111] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:38.262156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c108310-4a97-4ea0-ac55-b13c378cc59f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:38.266081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1055/2112] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:38.267043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03305def-f2c3-4147-bb0c-3ad38d05ab09 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:38.282899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6c108310-4a97-4ea0-ac55-b13c378cc59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:38.283058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03305def-f2c3-4147-bb0c-3ad38d05ab09 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:38.287304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d76779-08fb-45cb-8847-1d4ae2687de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:38.288553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d76779-08fb-45cb-8847-1d4ae2687de6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:38.289124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d76779-08fb-45cb-8847-1d4ae2687de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:38.289586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d76779-08fb-45cb-8847-1d4ae2687de6 admin admin] PATH_INFO: `/v3/projects/46383422ee4d44fdab36c49f904b8440/users/bcb5e00d77924bc5afe7dc77cefee8ee/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:38.304507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c108310-4a97-4ea0-ac55-b13c378cc59f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:38.310592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:38.311304 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:38.311618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:38.311979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] PATH_INFO: `/v3/users/4f0a1e4e0d364927858eddbc38d13404` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:38.313231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0a1e4e0d364927858eddbc38d13404)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:38.314233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d76779-08fb-45cb-8847-1d4ae2687de6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46383422ee4d44fdab36c49f904b8440, user_id=bcb5e00d77924bc5afe7dc77cefee8ee, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:38.315415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:38.317771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d76779-08fb-45cb-8847-1d4ae2687de6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:38.446942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1056/2113] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:38 2022] PUT /identity/v3/projects/46383422ee4d44fdab36c49f904b8440/users/bcb5e00d77924bc5afe7dc77cefee8ee/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:38.448556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d76779-08fb-45cb-8847-1d4ae2687de6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:38.450810 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c8d76779-08fb-45cb-8847-1d4ae2687de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:38.552450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d76779-08fb-45cb-8847-1d4ae2687de6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:38.556946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6812b4-78cf-4b0c-b2b8-a63a4bb14e2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:38.557677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6812b4-78cf-4b0c-b2b8-a63a4bb14e2a service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:38.558118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6812b4-78cf-4b0c-b2b8-a63a4bb14e2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:38.558397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6812b4-78cf-4b0c-b2b8-a63a4bb14e2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:38.633555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6812b4-78cf-4b0c-b2b8-a63a4bb14e2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:38.633555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6812b4-78cf-4b0c-b2b8-a63a4bb14e2a service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:38.644213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1057/2114] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:38.645210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6812b4-78cf-4b0c-b2b8-a63a4bb14e2a service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:38.760978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0a1e4e0d364927858eddbc38d13404'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:38.775964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6812b4-78cf-4b0c-b2b8-a63a4bb14e2a service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:38.784380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda1c85-f82b-476d-b496-d92c4999e9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:38.799016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda1c85-f82b-476d-b496-d92c4999e9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:38.799734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda1c85-f82b-476d-b496-d92c4999e9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:38.800233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda1c85-f82b-476d-b496-d92c4999e9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:38.801437 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda1c85-f82b-476d-b496-d92c4999e9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:38.804900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda1c85-f82b-476d-b496-d92c4999e9d0 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:38.837588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1058/2115] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:38.839031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda1c85-f82b-476d-b496-d92c4999e9d0 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:38.843961 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-adda1c85-f82b-476d-b496-d92c4999e9d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:38.859020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda1c85-f82b-476d-b496-d92c4999e9d0 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:38.868791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:38.869745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:38.870220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:38.870944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] PATH_INFO: `/v3/projects/f6556ad62de2429b8f98b3f6432192fd` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:38.872907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6556ad62de2429b8f98b3f6432192fd)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:38.878666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:38.914158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0a1e4e0d364927858eddbc38d13404'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:38.916014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0a1e4e0d364927858eddbc38d13404'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:38.949990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0a1e4e0d364927858eddbc38d13404'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:38.950170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0a1e4e0d364927858eddbc38d13404'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:38.986254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1058/2116] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:38 2022] DELETE /identity/v3/users/4f0a1e4e0d364927858eddbc38d13404 => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:38.988009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:38.990539 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:39.084074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6556ad62de2429b8f98b3f6432192fd', 'request_id': 'req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:39.091947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8e74d-a416-4199-abea-f60db1096256 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:39.093191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6556ad62de2429b8f98b3f6432192fd', 'request_id': 'req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:39.165421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d1808-2ff3-48a0-b091-bd482c54d936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:39.166007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d1808-2ff3-48a0-b091-bd482c54d936 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.166290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d1808-2ff3-48a0-b091-bd482c54d936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.166569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d1808-2ff3-48a0-b091-bd482c54d936 admin admin] PATH_INFO: `/v3/projects/46383422ee4d44fdab36c49f904b8440/users/bcb5e00d77924bc5afe7dc77cefee8ee/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.168638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d1808-2ff3-48a0-b091-bd482c54d936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46383422ee4d44fdab36c49f904b8440, user_id=bcb5e00d77924bc5afe7dc77cefee8ee, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:39.171409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d1808-2ff3-48a0-b091-bd482c54d936 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:39.177868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6556ad62de2429b8f98b3f6432192fd', 'request_id': 'req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:39.184755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1059/2117] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:38 2022] PUT /identity/v3/projects/46383422ee4d44fdab36c49f904b8440/users/bcb5e00d77924bc5afe7dc77cefee8ee/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:39.186767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d1808-2ff3-48a0-b091-bd482c54d936 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:39.189543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-321d1808-2ff3-48a0-b091-bd482c54d936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:39.207720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] The token cache is being invalidate because project f6556ad62de2429b8f98b3f6432192fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:39.208331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:39.209711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2998a2ce-ab78-4cf9-92dc-3423b8f012dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:39.211860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1059/2118] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:38 2022] DELETE /identity/v3/projects/f6556ad62de2429b8f98b3f6432192fd => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:39.214383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440daf8-38c9-47a3-827f-892ca7b440ee None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.214875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440daf8-38c9-47a3-827f-892ca7b440ee None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.215356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440daf8-38c9-47a3-827f-892ca7b440ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.248582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6440daf8-38c9-47a3-827f-892ca7b440ee None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:39.250986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6440daf8-38c9-47a3-827f-892ca7b440ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:39.272125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d1808-2ff3-48a0-b091-bd482c54d936 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:39.278812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea391d75-b56c-4c08-abfa-01a31e71d00f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:39.280001 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea391d75-b56c-4c08-abfa-01a31e71d00f service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.280445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea391d75-b56c-4c08-abfa-01a31e71d00f service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.280920 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea391d75-b56c-4c08-abfa-01a31e71d00f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.284318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ea391d75-b56c-4c08-abfa-01a31e71d00f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:39.332645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:39.358257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1060/2119] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:39.358257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565e42-f26e-4fe6-942a-04871642b83c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.358257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565e42-f26e-4fe6-942a-04871642b83c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.358257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23565e42-f26e-4fe6-942a-04871642b83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.373910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea391d75-b56c-4c08-abfa-01a31e71d00f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:39.375575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea391d75-b56c-4c08-abfa-01a31e71d00f service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:39.386406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1060/2120] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:39.387885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29a30-9963-4592-b8e4-057b60963485 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.393139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29a30-9963-4592-b8e4-057b60963485 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.393139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29a30-9963-4592-b8e4-057b60963485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.393700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-23565e42-f26e-4fe6-942a-04871642b83c None None] MFA Rules not processed for user `bcb5e00d77924bc5afe7dc77cefee8ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:39.395889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-23565e42-f26e-4fe6-942a-04871642b83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:39.417400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-0eb29a30-9963-4592-b8e4-057b60963485 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:39.418689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0eb29a30-9963-4592-b8e4-057b60963485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:39.429346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1061/2121] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:39.429959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29a30-9963-4592-b8e4-057b60963485 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:39.431669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0eb29a30-9963-4592-b8e4-057b60963485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:39.522380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29a30-9963-4592-b8e4-057b60963485 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:39.524372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1061/2122] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:39.527149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea447b-6fe0-4731-a05a-0ff5bc78d6f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:39.527381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fcc5d-c8c9-4c72-9342-4d8d8df9dcbd None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.527729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fcc5d-c8c9-4c72-9342-4d8d8df9dcbd None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.527915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea447b-6fe0-4731-a05a-0ff5bc78d6f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.528317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea447b-6fe0-4731-a05a-0ff5bc78d6f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.528469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fcc5d-c8c9-4c72-9342-4d8d8df9dcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.528612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea447b-6fe0-4731-a05a-0ff5bc78d6f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.542345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea447b-6fe0-4731-a05a-0ff5bc78d6f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:39.545802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea447b-6fe0-4731-a05a-0ff5bc78d6f1 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:39.556580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f53fcc5d-c8c9-4c72-9342-4d8d8df9dcbd None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:39.557319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f53fcc5d-c8c9-4c72-9342-4d8d8df9dcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:39.565396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1062/2123] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:39.568622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1062/2124] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:39.575053 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea447b-6fe0-4731-a05a-0ff5bc78d6f1 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:39.586426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea447b-6fe0-4731-a05a-0ff5bc78d6f1 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:39.591928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63a585-4d41-41bc-b2cf-ca256ac10e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:39.592363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63a585-4d41-41bc-b2cf-ca256ac10e1e admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.592646 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63a585-4d41-41bc-b2cf-ca256ac10e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.592923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63a585-4d41-41bc-b2cf-ca256ac10e1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.593591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63a585-4d41-41bc-b2cf-ca256ac10e1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:39.593896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63a585-4d41-41bc-b2cf-ca256ac10e1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:39.595732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63a585-4d41-41bc-b2cf-ca256ac10e1e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:39.612191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1063/2125] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:31:39.614258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fcc5d-c8c9-4c72-9342-4d8d8df9dcbd None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:39.624669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fcc5d-c8c9-4c72-9342-4d8d8df9dcbd None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:39.630215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d2962-d2d3-443d-a7c2-ad2cf098e322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:39.630850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d2962-d2d3-443d-a7c2-ad2cf098e322 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.631131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d2962-d2d3-443d-a7c2-ad2cf098e322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.631477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d2962-d2d3-443d-a7c2-ad2cf098e322 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.632115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d2962-d2d3-443d-a7c2-ad2cf098e322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:39.634082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d2962-d2d3-443d-a7c2-ad2cf098e322 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:39.812254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1063/2126] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:39 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:39.817457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63a585-4d41-41bc-b2cf-ca256ac10e1e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:39.844663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63a585-4d41-41bc-b2cf-ca256ac10e1e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:39.849210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d2962-d2d3-443d-a7c2-ad2cf098e322 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:39.854346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-835d2962-d2d3-443d-a7c2-ad2cf098e322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:39.947103 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d2962-d2d3-443d-a7c2-ad2cf098e322 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:39.951441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e531-c0bf-4460-9c00-a9ac069c67c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:39.952281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e531-c0bf-4460-9c00-a9ac069c67c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.952598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e531-c0bf-4460-9c00-a9ac069c67c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.952928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e531-c0bf-4460-9c00-a9ac069c67c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.955684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d5b4e531-c0bf-4460-9c00-a9ac069c67c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:39.964174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546fadd-2aad-4010-ad20-85607e676f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:39.965211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546fadd-2aad-4010-ad20-85607e676f0c admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:39.965710 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546fadd-2aad-4010-ad20-85607e676f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:39.966188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546fadd-2aad-4010-ad20-85607e676f0c admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:39.968205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546fadd-2aad-4010-ad20-85607e676f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:39.972137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546fadd-2aad-4010-ad20-85607e676f0c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:40.074610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4e531-c0bf-4460-9c00-a9ac069c67c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:40.077584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4e531-c0bf-4460-9c00-a9ac069c67c1 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:40.096357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1064/2127] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:40.118962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1064/2128] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:39 2022] POST /identity/v3/users => generated 284 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:40.123759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e531-c0bf-4460-9c00-a9ac069c67c1 service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:40.136255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e531-c0bf-4460-9c00-a9ac069c67c1 service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:40.142364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12b24a-03f4-4cd6-82d8-482c71454648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:40.143070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12b24a-03f4-4cd6-82d8-482c71454648 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:40.143353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12b24a-03f4-4cd6-82d8-482c71454648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:40.143668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12b24a-03f4-4cd6-82d8-482c71454648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:40.144396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12b24a-03f4-4cd6-82d8-482c71454648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:40.146047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12b24a-03f4-4cd6-82d8-482c71454648 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:40.165830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546fadd-2aad-4010-ad20-85607e676f0c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:40.168803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5546fadd-2aad-4010-ad20-85607e676f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:40.169361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1065/2129] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:40.175680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12b24a-03f4-4cd6-82d8-482c71454648 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:40.196174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546fadd-2aad-4010-ad20-85607e676f0c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:40.206017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:40.207238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:40.207574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:40.208264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] PATH_INFO: `/v3/projects/c2a148e97ebd4a88b83747b5ef0773e8` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:40.210747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2a148e97ebd4a88b83747b5ef0773e8)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:40.214701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12b24a-03f4-4cd6-82d8-482c71454648 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:40.215534 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:40.233183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedffc12-a579-4d12-813f-3ea8742f3bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:40.233871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedffc12-a579-4d12-813f-3ea8742f3bdb admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:40.233871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedffc12-a579-4d12-813f-3ea8742f3bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:40.234034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedffc12-a579-4d12-813f-3ea8742f3bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:40.234912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedffc12-a579-4d12-813f-3ea8742f3bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:40.237965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedffc12-a579-4d12-813f-3ea8742f3bdb admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:40.260906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1066/2130] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:40.266699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedffc12-a579-4d12-813f-3ea8742f3bdb admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:40.285443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedffc12-a579-4d12-813f-3ea8742f3bdb admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:40.293226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd30b5d-dfa8-4299-b9ce-3fbb5a30e0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:40.294199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30b5d-dfa8-4299-b9ce-3fbb5a30e0ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:40.294689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30b5d-dfa8-4299-b9ce-3fbb5a30e0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:40.295117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd30b5d-dfa8-4299-b9ce-3fbb5a30e0ab admin admin] PATH_INFO: `/v3/projects/d9adff9463f34ccab428829a3e3b32b0/users/a129445ddd474607852bf8c4f8cee59c/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:40.325232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd30b5d-dfa8-4299-b9ce-3fbb5a30e0ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9adff9463f34ccab428829a3e3b32b0, user_id=a129445ddd474607852bf8c4f8cee59c, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:40.329580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd30b5d-dfa8-4299-b9ce-3fbb5a30e0ab admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:40.497563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1067/2131] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:40 2022] PUT /identity/v3/projects/d9adff9463f34ccab428829a3e3b32b0/users/a129445ddd474607852bf8c4f8cee59c/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:40.510677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547050-f6bc-4d15-90cc-efa8716b86e2 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:40.510809 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2a148e97ebd4a88b83747b5ef0773e8', 'request_id': 'req-8c72259a-30f8-4f25-838f-d956e6c00354'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:40.511141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547050-f6bc-4d15-90cc-efa8716b86e2 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:40.511487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547050-f6bc-4d15-90cc-efa8716b86e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:40.521566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2a148e97ebd4a88b83747b5ef0773e8', 'request_id': 'req-8c72259a-30f8-4f25-838f-d956e6c00354'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:40.564766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ec547050-f6bc-4d15-90cc-efa8716b86e2 None None] MFA Rules not processed for user `a129445ddd474607852bf8c4f8cee59c`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:40.566787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ec547050-f6bc-4d15-90cc-efa8716b86e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:40.637208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2a148e97ebd4a88b83747b5ef0773e8', 'request_id': 'req-8c72259a-30f8-4f25-838f-d956e6c00354'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:40.656696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] The token cache is being invalidate because project c2a148e97ebd4a88b83747b5ef0773e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:40.656696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:40.657528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c72259a-30f8-4f25-838f-d956e6c00354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:40.663180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1065/2132] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:40 2022] DELETE /identity/v3/projects/c2a148e97ebd4a88b83747b5ef0773e8 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:40.672161 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080785-56a5-4923-bfec-6ee3af9c42b8 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:40.672634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080785-56a5-4923-bfec-6ee3af9c42b8 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:40.673111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59080785-56a5-4923-bfec-6ee3af9c42b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:40.678979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1068/2133] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:40.684970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547050-f6bc-4d15-90cc-efa8716b86e2 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:40.722867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-59080785-56a5-4923-bfec-6ee3af9c42b8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:40.724531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-59080785-56a5-4923-bfec-6ee3af9c42b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:40.801950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547050-f6bc-4d15-90cc-efa8716b86e2 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:40.816022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ed556-e179-466b-9982-295d6012a9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:40.816022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ed556-e179-466b-9982-295d6012a9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:40.816022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ed556-e179-466b-9982-295d6012a9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:40.816022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ed556-e179-466b-9982-295d6012a9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:40.816553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ed556-e179-466b-9982-295d6012a9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:40.817957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ed556-e179-466b-9982-295d6012a9d3 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:40.835574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1066/2134] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:40.837520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947b0fe-fabf-4d9d-884c-7e88782fa7a0 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:40.837983 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947b0fe-fabf-4d9d-884c-7e88782fa7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:40.839076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947b0fe-fabf-4d9d-884c-7e88782fa7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:40.846124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:40.872295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e947b0fe-fabf-4d9d-884c-7e88782fa7a0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:40.875202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e947b0fe-fabf-4d9d-884c-7e88782fa7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:40.893753 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1067/2135] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:40.895734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947b0fe-fabf-4d9d-884c-7e88782fa7a0 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:40.911718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947b0fe-fabf-4d9d-884c-7e88782fa7a0 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:40.917721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5e1ef-5193-4264-a651-ea5df2b1e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:40.918764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5e1ef-5193-4264-a651-ea5df2b1e7cf admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:40.919267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5e1ef-5193-4264-a651-ea5df2b1e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:40.920308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5e1ef-5193-4264-a651-ea5df2b1e7cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:40.921563 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5e1ef-5193-4264-a651-ea5df2b1e7cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:40.922117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5e1ef-5193-4264-a651-ea5df2b1e7cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:40.925341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5e1ef-5193-4264-a651-ea5df2b1e7cf admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:40.946152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1068/2136] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:40.951799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5e1ef-5193-4264-a651-ea5df2b1e7cf admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:40.963967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5e1ef-5193-4264-a651-ea5df2b1e7cf admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:40.969742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6aaf9-62fd-4c3f-a64b-44d7e26f7fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:40.970578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6aaf9-62fd-4c3f-a64b-44d7e26f7fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:40.970677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6aaf9-62fd-4c3f-a64b-44d7e26f7fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:40.970893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac6aaf9-62fd-4c3f-a64b-44d7e26f7fa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:40.971592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6aaf9-62fd-4c3f-a64b-44d7e26f7fa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:40.973761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac6aaf9-62fd-4c3f-a64b-44d7e26f7fa4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:40.995165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1069/2137] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:40 2022] POST /identity/v3/projects => generated 367 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:40.999445 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ed556-e179-466b-9982-295d6012a9d3 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:41.001796 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-078ed556-e179-466b-9982-295d6012a9d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:41.112634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1069/2138] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:40 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:41.112847 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ed556-e179-466b-9982-295d6012a9d3 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:41.114841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6aaf9-62fd-4c3f-a64b-44d7e26f7fa4 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:41.182763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f3070-8ae2-438a-b922-2b03eec00a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:41.183475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f3070-8ae2-438a-b922-2b03eec00a16 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:41.183850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f3070-8ae2-438a-b922-2b03eec00a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:41.184202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f3070-8ae2-438a-b922-2b03eec00a16 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:41.185067 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f3070-8ae2-438a-b922-2b03eec00a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:41.187701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f3070-8ae2-438a-b922-2b03eec00a16 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:41.225781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac6aaf9-62fd-4c3f-a64b-44d7e26f7fa4 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:41.230899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10bbd-7aa2-4cb5-8796-ffea02367851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:41.231721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10bbd-7aa2-4cb5-8796-ffea02367851 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:41.232009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10bbd-7aa2-4cb5-8796-ffea02367851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:41.232922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10bbd-7aa2-4cb5-8796-ffea02367851 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:41.233581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d10bbd-7aa2-4cb5-8796-ffea02367851 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:41.235586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d10bbd-7aa2-4cb5-8796-ffea02367851 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:41.346513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1070/2139] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:40 2022] POST /identity/v3/users => generated 290 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:41.355931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f3070-8ae2-438a-b922-2b03eec00a16 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:41.400543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f3070-8ae2-438a-b922-2b03eec00a16 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:41.402391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1070/2140] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:41 2022] POST /identity/v3/users => generated 286 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:41.409030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10bbd-7aa2-4cb5-8796-ffea02367851 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:41.415102 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074aedd-1bbe-43f2-8759-483658a934b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:41.418911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074aedd-1bbe-43f2-8759-483658a934b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:41.419429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074aedd-1bbe-43f2-8759-483658a934b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:41.420588 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074aedd-1bbe-43f2-8759-483658a934b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:41.421897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074aedd-1bbe-43f2-8759-483658a934b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:41.425252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074aedd-1bbe-43f2-8759-483658a934b1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:41.435845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10bbd-7aa2-4cb5-8796-ffea02367851 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:41.442945 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4eda4-ad6a-48c3-aca4-ef32fa0665c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:41.459910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4eda4-ad6a-48c3-aca4-ef32fa0665c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:41.459910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4eda4-ad6a-48c3-aca4-ef32fa0665c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:41.459910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4eda4-ad6a-48c3-aca4-ef32fa0665c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:41.459910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4eda4-ad6a-48c3-aca4-ef32fa0665c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:41.459910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4eda4-ad6a-48c3-aca4-ef32fa0665c5 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:41.460727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1071/2141] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:41.461120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074aedd-1bbe-43f2-8759-483658a934b1 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:41.463606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1071/2142] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:41.469643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4eda4-ad6a-48c3-aca4-ef32fa0665c5 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:41.490324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4eda4-ad6a-48c3-aca4-ef32fa0665c5 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:41.492815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074aedd-1bbe-43f2-8759-483658a934b1 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:41.498809 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2510f-05a7-4601-9191-adffc7ab414e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:41.512357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2510f-05a7-4601-9191-adffc7ab414e admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:41.512357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2510f-05a7-4601-9191-adffc7ab414e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:41.512357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2510f-05a7-4601-9191-adffc7ab414e admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:41.513057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b25a50-88f8-4418-8165-f0d8f6a74362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:41.513057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b25a50-88f8-4418-8165-f0d8f6a74362 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:41.513504 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2510f-05a7-4601-9191-adffc7ab414e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:41.514077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b25a50-88f8-4418-8165-f0d8f6a74362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:41.514077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b25a50-88f8-4418-8165-f0d8f6a74362 admin admin] PATH_INFO: `/v3/projects/ebc65d4e3d42431db5d52483451a0ae0/users/15031c9616cc437ebf69f41d9b325a94/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:41.514249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2510f-05a7-4601-9191-adffc7ab414e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:41.531994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1072/2143] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:41.534308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2510f-05a7-4601-9191-adffc7ab414e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:41.550523 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2510f-05a7-4601-9191-adffc7ab414e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:41.551135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b25a50-88f8-4418-8165-f0d8f6a74362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc65d4e3d42431db5d52483451a0ae0, user_id=15031c9616cc437ebf69f41d9b325a94, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:41.564247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b25a50-88f8-4418-8165-f0d8f6a74362 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:41.564541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5c3f7-c0f8-4c0b-83c4-4b9403465e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:41.565463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5c3f7-c0f8-4c0b-83c4-4b9403465e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:41.566060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5c3f7-c0f8-4c0b-83c4-4b9403465e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:41.566964 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5c3f7-c0f8-4c0b-83c4-4b9403465e63 admin admin] PATH_INFO: `/v3/projects/3816d889342e400383f7e5e6f4ebbe04/users/1658e3b405ea41939ff42b6bb36d87f7/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:41.599306 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5c3f7-c0f8-4c0b-83c4-4b9403465e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3816d889342e400383f7e5e6f4ebbe04, user_id=1658e3b405ea41939ff42b6bb36d87f7, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:41.603020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5c3f7-c0f8-4c0b-83c4-4b9403465e63 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:41.760316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1072/2144] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:41 2022] PUT /identity/v3/projects/ebc65d4e3d42431db5d52483451a0ae0/users/15031c9616cc437ebf69f41d9b325a94/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:41.761110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1073/2145] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:41 2022] PUT /identity/v3/projects/3816d889342e400383f7e5e6f4ebbe04/users/1658e3b405ea41939ff42b6bb36d87f7/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:41.761365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b25a50-88f8-4418-8165-f0d8f6a74362 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:41.773943 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a1018-51c6-4ffd-9455-5d8e6d6e5281 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:41.775817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a1018-51c6-4ffd-9455-5d8e6d6e5281 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:41.776276 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a1018-51c6-4ffd-9455-5d8e6d6e5281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:41.815094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8b8a1018-51c6-4ffd-9455-5d8e6d6e5281 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:41.822285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8b8a1018-51c6-4ffd-9455-5d8e6d6e5281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:41.865858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b25a50-88f8-4418-8165-f0d8f6a74362 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:41.931153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1074/2146] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:41.945353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a1018-51c6-4ffd-9455-5d8e6d6e5281 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:41.947537 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8b8a1018-51c6-4ffd-9455-5d8e6d6e5281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:41.963316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40999942-502c-4f3f-8cb7-56802653e68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:41.965537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40999942-502c-4f3f-8cb7-56802653e68b admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:41.965537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40999942-502c-4f3f-8cb7-56802653e68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:41.966062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40999942-502c-4f3f-8cb7-56802653e68b admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:41.967751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40999942-502c-4f3f-8cb7-56802653e68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:41.971232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40999942-502c-4f3f-8cb7-56802653e68b admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:42.013253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1073/2147] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:42.017351 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40999942-502c-4f3f-8cb7-56802653e68b admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:42.029770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:42.054836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40999942-502c-4f3f-8cb7-56802653e68b admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:42.063511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa7879-729b-4ab6-b7c2-80a4922b7537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:42.065483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa7879-729b-4ab6-b7c2-80a4922b7537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:42.066087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa7879-729b-4ab6-b7c2-80a4922b7537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:42.066585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa7879-729b-4ab6-b7c2-80a4922b7537 admin admin] PATH_INFO: `/v3/projects/ebc65d4e3d42431db5d52483451a0ae0/users/15031c9616cc437ebf69f41d9b325a94/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:42.070417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa7879-729b-4ab6-b7c2-80a4922b7537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc65d4e3d42431db5d52483451a0ae0, user_id=15031c9616cc437ebf69f41d9b325a94, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:42.079001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa7879-729b-4ab6-b7c2-80a4922b7537 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:42.086791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a1018-51c6-4ffd-9455-5d8e6d6e5281 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:42.095278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f4dcd-1826-4ea3-a238-6e464e136e51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:42.096239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f4dcd-1826-4ea3-a238-6e464e136e51 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:42.096698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f4dcd-1826-4ea3-a238-6e464e136e51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:42.097131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f4dcd-1826-4ea3-a238-6e464e136e51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:42.104268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1074/2148] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:42 2022] PUT /identity/v3/projects/ebc65d4e3d42431db5d52483451a0ae0/users/15031c9616cc437ebf69f41d9b325a94/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:42.128250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f4dcd-1826-4ea3-a238-6e464e136e51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:42.130520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f4dcd-1826-4ea3-a238-6e464e136e51 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:42.139652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0ade1-b11b-403c-9b6a-a7dfa0f926a4 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:42.141088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0ade1-b11b-403c-9b6a-a7dfa0f926a4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:42.141840 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0ade1-b11b-403c-9b6a-a7dfa0f926a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:42.148272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1075/2149] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:42.196175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-fdd0ade1-b11b-403c-9b6a-a7dfa0f926a4 None None] MFA Rules not processed for user `15031c9616cc437ebf69f41d9b325a94`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:42.198402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-fdd0ade1-b11b-403c-9b6a-a7dfa0f926a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:42.360448 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1075/2150] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:42.378507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f4dcd-1826-4ea3-a238-6e464e136e51 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:42.380291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-285f4dcd-1826-4ea3-a238-6e464e136e51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:42.483240 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f4dcd-1826-4ea3-a238-6e464e136e51 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:42.560508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:42.569410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e0f64-ee7c-4f6d-a5ac-9b041e7853e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:42.570903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e0f64-ee7c-4f6d-a5ac-9b041e7853e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:42.571415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e0f64-ee7c-4f6d-a5ac-9b041e7853e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:42.571950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23e0f64-ee7c-4f6d-a5ac-9b041e7853e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:42.590151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e0f64-ee7c-4f6d-a5ac-9b041e7853e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:42.593976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23e0f64-ee7c-4f6d-a5ac-9b041e7853e5 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:42.626044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1076/2151] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:43.087203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0ade1-b11b-403c-9b6a-a7dfa0f926a4 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:43.104503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0ade1-b11b-403c-9b6a-a7dfa0f926a4 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:43.115699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253053b3-57ae-4b31-a0ac-9378811b850e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:43.116500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253053b3-57ae-4b31-a0ac-9378811b850e service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:43.117357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253053b3-57ae-4b31-a0ac-9378811b850e service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:43.117685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253053b3-57ae-4b31-a0ac-9378811b850e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:43.119882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-253053b3-57ae-4b31-a0ac-9378811b850e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:43.185168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e0f64-ee7c-4f6d-a5ac-9b041e7853e5 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:43.191046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c23e0f64-ee7c-4f6d-a5ac-9b041e7853e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:43.253261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253053b3-57ae-4b31-a0ac-9378811b850e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:43.255197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253053b3-57ae-4b31-a0ac-9378811b850e service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:43.283642 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1076/2152] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:43.343786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23e0f64-ee7c-4f6d-a5ac-9b041e7853e5 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:43.356959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:43.358461 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:43.361691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:43.362904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] PATH_INFO: `/v3/users/43dbde949334471fbf157054a070bcfe` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:43.365645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43dbde949334471fbf157054a070bcfe)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:43.370095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:43.468562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253053b3-57ae-4b31-a0ac-9378811b850e service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:43.470610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-253053b3-57ae-4b31-a0ac-9378811b850e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:43.494222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253053b3-57ae-4b31-a0ac-9378811b850e service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:43.500185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7eded2-ae32-4871-b4b4-30463f00e0bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:43.501029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eded2-ae32-4871-b4b4-30463f00e0bf service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:43.501385 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eded2-ae32-4871-b4b4-30463f00e0bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:43.501903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7eded2-ae32-4871-b4b4-30463f00e0bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:43.504500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-cc7eded2-ae32-4871-b4b4-30463f00e0bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:43.599227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7eded2-ae32-4871-b4b4-30463f00e0bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:43.603129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7eded2-ae32-4871-b4b4-30463f00e0bf service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:43.620008 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1077/2153] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:43.794250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43dbde949334471fbf157054a070bcfe'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:43.895207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43dbde949334471fbf157054a070bcfe'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:43.895946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43dbde949334471fbf157054a070bcfe'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:43.917650 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43dbde949334471fbf157054a070bcfe'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:43.919329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43dbde949334471fbf157054a070bcfe'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:43.943704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1077/2154] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:43 2022] DELETE /identity/v3/users/43dbde949334471fbf157054a070bcfe => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:43.949843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43369a74-e49f-45bb-afc3-8abfc8c65cdf None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:43.950131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43369a74-e49f-45bb-afc3-8abfc8c65cdf None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:43.957693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43369a74-e49f-45bb-afc3-8abfc8c65cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:43.978022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-43369a74-e49f-45bb-afc3-8abfc8c65cdf None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:43.979407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-43369a74-e49f-45bb-afc3-8abfc8c65cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:44.064862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1078/2155] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:44.080661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:44.082532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:44.152798 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60591eb-448b-44f5-adf3-94851e87f029 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:44.158930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1cb47-0676-44df-b447-0613e6bc07eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:44.160036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1cb47-0676-44df-b447-0613e6bc07eb service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:44.160525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1cb47-0676-44df-b447-0613e6bc07eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:44.161021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1cb47-0676-44df-b447-0613e6bc07eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:44.173581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1cb47-0676-44df-b447-0613e6bc07eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:44.176528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1cb47-0676-44df-b447-0613e6bc07eb service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:44.190080 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1078/2156] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:44.535860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf750b4-730f-4b03-828a-bf54c7daca22 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:44.537973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf750b4-730f-4b03-828a-bf54c7daca22 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:44.538594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf750b4-730f-4b03-828a-bf54c7daca22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:44.581083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-faf750b4-730f-4b03-828a-bf54c7daca22 None None] MFA Rules not processed for user `1658e3b405ea41939ff42b6bb36d87f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:44.581973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-faf750b4-730f-4b03-828a-bf54c7daca22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:44.714468 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1079/2157] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:44.735326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:44.746722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1cb47-0676-44df-b447-0613e6bc07eb service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:44.748312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-72b1cb47-0676-44df-b447-0613e6bc07eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:44.814255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1cb47-0676-44df-b447-0613e6bc07eb service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:44.818509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047b1dd-00fd-46e1-bfcc-c84045f9a65a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:44.819179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047b1dd-00fd-46e1-bfcc-c84045f9a65a service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:44.819769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047b1dd-00fd-46e1-bfcc-c84045f9a65a service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:44.820043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4047b1dd-00fd-46e1-bfcc-c84045f9a65a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:44.834367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047b1dd-00fd-46e1-bfcc-c84045f9a65a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:44.836742 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf750b4-730f-4b03-828a-bf54c7daca22 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:44.837121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4047b1dd-00fd-46e1-bfcc-c84045f9a65a service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:44.839259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-faf750b4-730f-4b03-828a-bf54c7daca22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:44.850067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1079/2158] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:44.886699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf750b4-730f-4b03-828a-bf54c7daca22 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:44.902023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:44.902700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:44.902995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:44.903342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] PATH_INFO: `/v3/projects/9b86db5b3dc3461a803e8541509ed2fb` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:44.905789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b86db5b3dc3461a803e8541509ed2fb)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:44.908089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:45.132177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b86db5b3dc3461a803e8541509ed2fb', 'request_id': 'req-7af42b45-d4cf-477c-b521-73de93ce5ef8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:45.143161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b86db5b3dc3461a803e8541509ed2fb', 'request_id': 'req-7af42b45-d4cf-477c-b521-73de93ce5ef8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:45.244068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b86db5b3dc3461a803e8541509ed2fb', 'request_id': 'req-7af42b45-d4cf-477c-b521-73de93ce5ef8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:45.256870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:45.266901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] The token cache is being invalidate because project 9b86db5b3dc3461a803e8541509ed2fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:45.267512 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:45.268668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:45.270289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1080/2159] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:44 2022] DELETE /identity/v3/projects/9b86db5b3dc3461a803e8541509ed2fb => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:45.275173 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047b1dd-00fd-46e1-bfcc-c84045f9a65a service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:45.288358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4047b1dd-00fd-46e1-bfcc-c84045f9a65a service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:45.369397 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:45.370755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:45.371071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:45.371624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] PATH_INFO: `/v3/users/f27281d4d3eb4933873a1547f69110cc` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:45.374416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f27281d4d3eb4933873a1547f69110cc)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:45.377830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:45.760400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f27281d4d3eb4933873a1547f69110cc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:45.900469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f27281d4d3eb4933873a1547f69110cc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:45.904951 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f27281d4d3eb4933873a1547f69110cc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:45.921352 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f27281d4d3eb4933873a1547f69110cc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:45.922152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f27281d4d3eb4933873a1547f69110cc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:45.936008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1080/2160] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:45 2022] DELETE /identity/v3/users/f27281d4d3eb4933873a1547f69110cc => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:46.200532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:46.505875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:46.507648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:46.581169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af42b45-d4cf-477c-b521-73de93ce5ef8 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:46.644730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7bcc7b-23f1-4e22-b883-9f5f26d01e23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:46.645352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bcc7b-23f1-4e22-b883-9f5f26d01e23 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:46.645861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bcc7b-23f1-4e22-b883-9f5f26d01e23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:46.646235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7bcc7b-23f1-4e22-b883-9f5f26d01e23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:46.649612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4d7bcc7b-23f1-4e22-b883-9f5f26d01e23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:46.682867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:46.686006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:46.757513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bcc7b-23f1-4e22-b883-9f5f26d01e23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:46.760736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7bcc7b-23f1-4e22-b883-9f5f26d01e23 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:46.783533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1081/2161] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:46.812247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c13a0a-6691-4f20-b958-dae1f458ef34 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:46.818552 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:46.819396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:46.819728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:46.820195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] PATH_INFO: `/v3/projects/c26e334cbb7f48c5834f8acc68313043` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:46.821967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26e334cbb7f48c5834f8acc68313043)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:46.824984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:46.999567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26e334cbb7f48c5834f8acc68313043', 'request_id': 'req-27650121-d7e1-4e4d-899b-00b20ffa8df7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:47.005491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26e334cbb7f48c5834f8acc68313043', 'request_id': 'req-27650121-d7e1-4e4d-899b-00b20ffa8df7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:47.131262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26e334cbb7f48c5834f8acc68313043', 'request_id': 'req-27650121-d7e1-4e4d-899b-00b20ffa8df7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:47.167880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] The token cache is being invalidate because project c26e334cbb7f48c5834f8acc68313043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:47.167880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:47.168313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27650121-d7e1-4e4d-899b-00b20ffa8df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:47.172020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1081/2162] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:46 2022] DELETE /identity/v3/projects/c26e334cbb7f48c5834f8acc68313043 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:47.177339 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71bd9c-5292-4e83-9c51-d2cab7122684 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:47.177635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71bd9c-5292-4e83-9c51-d2cab7122684 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:47.178017 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71bd9c-5292-4e83-9c51-d2cab7122684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:47.182815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470aac9-7597-42f2-9802-5cc1de7ea0a3 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:47.183451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470aac9-7597-42f2-9802-5cc1de7ea0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:47.183965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470aac9-7597-42f2-9802-5cc1de7ea0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:47.200324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6d71bd9c-5292-4e83-9c51-d2cab7122684 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:47.202982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6d71bd9c-5292-4e83-9c51-d2cab7122684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:47.225318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-4470aac9-7597-42f2-9802-5cc1de7ea0a3 None None] MFA Rules not processed for user `1658e3b405ea41939ff42b6bb36d87f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:47.226756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4470aac9-7597-42f2-9802-5cc1de7ea0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:47.291734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1082/2163] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:47.294050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a0678-1d87-4454-ac5b-10dd0a5ef8ef None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:47.294236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a0678-1d87-4454-ac5b-10dd0a5ef8ef None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:47.294456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a0678-1d87-4454-ac5b-10dd0a5ef8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:47.294761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1082/2164] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:47.317572 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-965a0678-1d87-4454-ac5b-10dd0a5ef8ef None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:47.318973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-965a0678-1d87-4454-ac5b-10dd0a5ef8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:47.329541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1083/2165] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:47.334088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470aac9-7597-42f2-9802-5cc1de7ea0a3 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:47.346714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470aac9-7597-42f2-9802-5cc1de7ea0a3 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:47.352416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc33355-3622-406e-8436-2cc68476653f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:47.353044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc33355-3622-406e-8436-2cc68476653f admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:47.353335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc33355-3622-406e-8436-2cc68476653f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:47.353770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc33355-3622-406e-8436-2cc68476653f admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:47.354445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc33355-3622-406e-8436-2cc68476653f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:47.354792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc33355-3622-406e-8436-2cc68476653f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:47.356455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc33355-3622-406e-8436-2cc68476653f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:47.376021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1083/2166] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:47.378043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a0678-1d87-4454-ac5b-10dd0a5ef8ef None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:47.393399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a0678-1d87-4454-ac5b-10dd0a5ef8ef None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:47.421606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1d498-2319-41f1-b428-f36c86d14701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:47.423250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1d498-2319-41f1-b428-f36c86d14701 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:47.423715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1d498-2319-41f1-b428-f36c86d14701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:47.423851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1d498-2319-41f1-b428-f36c86d14701 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:47.424605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1d498-2319-41f1-b428-f36c86d14701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:47.430237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1d498-2319-41f1-b428-f36c86d14701 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:47.647506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1084/2167] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:47 2022] POST /identity/v3/projects => generated 387 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:47.648695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc33355-3622-406e-8436-2cc68476653f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:47.656206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:47.665047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc33355-3622-406e-8436-2cc68476653f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:47.756661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409392a5-ab92-4365-91fe-23cc72629651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:47.757343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409392a5-ab92-4365-91fe-23cc72629651 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:47.757626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409392a5-ab92-4365-91fe-23cc72629651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:47.757902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409392a5-ab92-4365-91fe-23cc72629651 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:47.758574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409392a5-ab92-4365-91fe-23cc72629651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:47.760687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409392a5-ab92-4365-91fe-23cc72629651 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:47.884549 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1084/2168] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:47 2022] POST /identity/v3/users => generated 294 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:47.888911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1d498-2319-41f1-b428-f36c86d14701 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:47.907022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1d498-2319-41f1-b428-f36c86d14701 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:47.916427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fcf42-4950-487a-a7b5-83aad8815485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:47.917875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fcf42-4950-487a-a7b5-83aad8815485 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:47.918259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fcf42-4950-487a-a7b5-83aad8815485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:47.918605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441fcf42-4950-487a-a7b5-83aad8815485 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:47.920148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fcf42-4950-487a-a7b5-83aad8815485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:47.922334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441fcf42-4950-487a-a7b5-83aad8815485 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:47.936404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1085/2169] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:47.941565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409392a5-ab92-4365-91fe-23cc72629651 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:47.955732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409392a5-ab92-4365-91fe-23cc72629651 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:47.962410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbcbeb-a774-4676-9226-2a6c4391ea5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:47.963517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbcbeb-a774-4676-9226-2a6c4391ea5d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:47.963789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbcbeb-a774-4676-9226-2a6c4391ea5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:47.966020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbcbeb-a774-4676-9226-2a6c4391ea5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:47.966020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbcbeb-a774-4676-9226-2a6c4391ea5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:47.966884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbcbeb-a774-4676-9226-2a6c4391ea5d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:47.981207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1085/2170] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:47.986262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fcf42-4950-487a-a7b5-83aad8815485 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:47.996736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441fcf42-4950-487a-a7b5-83aad8815485 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:48.002132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd9f3d-17f3-4258-b322-8969028687b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:48.002814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd9f3d-17f3-4258-b322-8969028687b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:48.003093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd9f3d-17f3-4258-b322-8969028687b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:48.003367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd9f3d-17f3-4258-b322-8969028687b6 admin admin] PATH_INFO: `/v3/projects/3ab13df5fb304cd5b1ea67c5c749a0b5/users/aec9d6116e354ec6a7565665320a57bf/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:48.020186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd9f3d-17f3-4258-b322-8969028687b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ab13df5fb304cd5b1ea67c5c749a0b5, user_id=aec9d6116e354ec6a7565665320a57bf, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:48.022884 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd9f3d-17f3-4258-b322-8969028687b6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:48.118953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1086/2171] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:47 2022] PUT /identity/v3/projects/3ab13df5fb304cd5b1ea67c5c749a0b5/users/aec9d6116e354ec6a7565665320a57bf/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:48.144147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92b0f5-47f1-4fcb-905c-97f10853e952 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:48.144507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92b0f5-47f1-4fcb-905c-97f10853e952 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:48.144880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92b0f5-47f1-4fcb-905c-97f10853e952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:48.179418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-5d92b0f5-47f1-4fcb-905c-97f10853e952 None None] MFA Rules not processed for user `aec9d6116e354ec6a7565665320a57bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:48.180673 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5d92b0f5-47f1-4fcb-905c-97f10853e952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:48.286569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1086/2172] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:48.291736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd9f3d-17f3-4258-b322-8969028687b6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:48.392131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd9f3d-17f3-4258-b322-8969028687b6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:48.418717 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada983f-c054-4c6f-8839-674fa2037550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:48.419522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada983f-c054-4c6f-8839-674fa2037550 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:48.419968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada983f-c054-4c6f-8839-674fa2037550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:48.420280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada983f-c054-4c6f-8839-674fa2037550 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:48.421258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada983f-c054-4c6f-8839-674fa2037550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:48.423255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada983f-c054-4c6f-8839-674fa2037550 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:48.501553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:48.541038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1087/2173] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:48 2022] POST /identity/v3/projects => generated 387 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:48.542779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92b0f5-47f1-4fcb-905c-97f10853e952 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:48.554847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92b0f5-47f1-4fcb-905c-97f10853e952 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:48.676427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e87667-6c61-4bb5-832a-006023edd3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:48.680945 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e87667-6c61-4bb5-832a-006023edd3bf admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:48.681873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e87667-6c61-4bb5-832a-006023edd3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:48.682525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e87667-6c61-4bb5-832a-006023edd3bf admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:48.683899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e87667-6c61-4bb5-832a-006023edd3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:48.687877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e87667-6c61-4bb5-832a-006023edd3bf admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:48.846509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1087/2174] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:48 2022] POST /identity/v3/users => generated 300 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:48.852808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada983f-c054-4c6f-8839-674fa2037550 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:48.868695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada983f-c054-4c6f-8839-674fa2037550 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:48.877243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d0ce1-2bde-4756-926f-5036377264ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:48.877596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d0ce1-2bde-4756-926f-5036377264ea admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:48.877891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d0ce1-2bde-4756-926f-5036377264ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:48.878041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d0ce1-2bde-4756-926f-5036377264ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:48.878721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d0ce1-2bde-4756-926f-5036377264ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:48.880428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d0ce1-2bde-4756-926f-5036377264ea admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:48.893267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1088/2175] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:48.898592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e87667-6c61-4bb5-832a-006023edd3bf admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:48.909707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e87667-6c61-4bb5-832a-006023edd3bf admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:48.916792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ffe10-0e57-4493-8a4d-5d1ea35d3be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:48.917956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ffe10-0e57-4493-8a4d-5d1ea35d3be0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:48.918441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ffe10-0e57-4493-8a4d-5d1ea35d3be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:48.918921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ffe10-0e57-4493-8a4d-5d1ea35d3be0 admin admin] PATH_INFO: `/v3/projects/e69767a828744c4996558d3e3089f967/users/174d381b7c1f4b06a65bea4274440b58/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:48.937751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ffe10-0e57-4493-8a4d-5d1ea35d3be0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69767a828744c4996558d3e3089f967, user_id=174d381b7c1f4b06a65bea4274440b58, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:48.942964 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ffe10-0e57-4493-8a4d-5d1ea35d3be0 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:49.066152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1088/2176] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:48 2022] PUT /identity/v3/projects/e69767a828744c4996558d3e3089f967/users/174d381b7c1f4b06a65bea4274440b58/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:49.073421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d0ce1-2bde-4756-926f-5036377264ea admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:49.086042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d0ce1-2bde-4756-926f-5036377264ea admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:49.166060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca590b-35db-4ae2-9378-51fee374dcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:49.166949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca590b-35db-4ae2-9378-51fee374dcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:49.167424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca590b-35db-4ae2-9378-51fee374dcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:49.167959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca590b-35db-4ae2-9378-51fee374dcd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:49.168835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca590b-35db-4ae2-9378-51fee374dcd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:49.172398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca590b-35db-4ae2-9378-51fee374dcd0 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:49.173690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:49.187292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1089/2177] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:49.190536 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ffe10-0e57-4493-8a4d-5d1ea35d3be0 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:49.221002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ffe10-0e57-4493-8a4d-5d1ea35d3be0 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:49.237718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc263-74a6-4094-8d54-85dc9fc2a185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:49.239147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc263-74a6-4094-8d54-85dc9fc2a185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:49.240247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc263-74a6-4094-8d54-85dc9fc2a185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:49.241148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dbc263-74a6-4094-8d54-85dc9fc2a185 admin admin] PATH_INFO: `/v3/projects/e69767a828744c4996558d3e3089f967/users/174d381b7c1f4b06a65bea4274440b58/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:49.244482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbc263-74a6-4094-8d54-85dc9fc2a185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69767a828744c4996558d3e3089f967, user_id=174d381b7c1f4b06a65bea4274440b58, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:49.249435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dbc263-74a6-4094-8d54-85dc9fc2a185 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:49.275670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1089/2178] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:49 2022] PUT /identity/v3/projects/e69767a828744c4996558d3e3089f967/users/174d381b7c1f4b06a65bea4274440b58/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:49.292435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cc460-5b05-46a7-a6ca-1f085a95b87d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:49.293185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cc460-5b05-46a7-a6ca-1f085a95b87d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:49.293185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cc460-5b05-46a7-a6ca-1f085a95b87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:49.351906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-db4cc460-5b05-46a7-a6ca-1f085a95b87d None None] MFA Rules not processed for user `174d381b7c1f4b06a65bea4274440b58`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:49.353337 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-db4cc460-5b05-46a7-a6ca-1f085a95b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:49.488200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1090/2179] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:49.505949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc263-74a6-4094-8d54-85dc9fc2a185 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:49.511541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a2dbc263-74a6-4094-8d54-85dc9fc2a185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:49.580031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dbc263-74a6-4094-8d54-85dc9fc2a185 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:49.585846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4b5b5-c284-4d6c-9b46-4a3bf919c4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:49.586490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b5b5-c284-4d6c-9b46-4a3bf919c4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:49.586778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b5b5-c284-4d6c-9b46-4a3bf919c4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:49.587049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b5b5-c284-4d6c-9b46-4a3bf919c4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:49.597896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4b5b5-c284-4d6c-9b46-4a3bf919c4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:49.602485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4b5b5-c284-4d6c-9b46-4a3bf919c4e8 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:49.611667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1090/2180] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:49.803430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cc460-5b05-46a7-a6ca-1f085a95b87d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:49.805892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:49.883980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cc460-5b05-46a7-a6ca-1f085a95b87d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:49.889172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:49.890092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:49.895430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:49.895430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] PATH_INFO: `/v3/users/aec9d6116e354ec6a7565665320a57bf` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:49.895430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec9d6116e354ec6a7565665320a57bf)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:49.895430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:50.174216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec9d6116e354ec6a7565665320a57bf'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:50.174881 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec9d6116e354ec6a7565665320a57bf'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:50.175497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec9d6116e354ec6a7565665320a57bf'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:50.183187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec9d6116e354ec6a7565665320a57bf'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:50.195870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec9d6116e354ec6a7565665320a57bf'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:50.322397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1091/2181] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:49 2022] DELETE /identity/v3/users/aec9d6116e354ec6a7565665320a57bf => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:50.329112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171dfd1-8c38-45ae-92cc-53c55c80e449 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:50.330016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171dfd1-8c38-45ae-92cc-53c55c80e449 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:50.330320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171dfd1-8c38-45ae-92cc-53c55c80e449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:50.362442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b171dfd1-8c38-45ae-92cc-53c55c80e449 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:50.363987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b171dfd1-8c38-45ae-92cc-53c55c80e449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:50.455695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1091/2182] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:50.470136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:50.472215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:50.555867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:50.566090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3c3a1-ffb7-48bb-8ba3-f142bae4db87 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:50.570325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7beef2-b131-4da0-8c41-d520e3e6d425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:50.570960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7beef2-b131-4da0-8c41-d520e3e6d425 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:50.571960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7beef2-b131-4da0-8c41-d520e3e6d425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:50.572249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7beef2-b131-4da0-8c41-d520e3e6d425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:50.585431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7beef2-b131-4da0-8c41-d520e3e6d425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:50.587695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7beef2-b131-4da0-8c41-d520e3e6d425 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:50.600149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1092/2183] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:50.736617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:51.188445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171dfd1-8c38-45ae-92cc-53c55c80e449 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:51.188445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b171dfd1-8c38-45ae-92cc-53c55c80e449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:51.221306 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171dfd1-8c38-45ae-92cc-53c55c80e449 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:51.236338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:51.237295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:51.237445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:51.237772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] PATH_INFO: `/v3/users/0e141315d9f7419686d355e7490bce3c` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:51.240025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e141315d9f7419686d355e7490bce3c)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:51.246210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:51.575712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e141315d9f7419686d355e7490bce3c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:51.651981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7beef2-b131-4da0-8c41-d520e3e6d425 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:51.653602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7c7beef2-b131-4da0-8c41-d520e3e6d425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:51.670791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7beef2-b131-4da0-8c41-d520e3e6d425 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:51.675862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:51.677129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:51.677758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:51.678345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] PATH_INFO: `/v3/projects/3ab13df5fb304cd5b1ea67c5c749a0b5` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:51.680136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ab13df5fb304cd5b1ea67c5c749a0b5)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:51.698150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e141315d9f7419686d355e7490bce3c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:51.699202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e141315d9f7419686d355e7490bce3c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:51.706838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:51.733891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e141315d9f7419686d355e7490bce3c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:51.736065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-003ffe5e-5bc2-4fdd-b8ba-8d725a4375cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e141315d9f7419686d355e7490bce3c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:51.773637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1092/2184] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:51 2022] DELETE /identity/v3/users/0e141315d9f7419686d355e7490bce3c => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:51.782205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c65812-ad95-4e08-8306-de1e85cc2c00 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:51.782961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c65812-ad95-4e08-8306-de1e85cc2c00 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:51.783456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c65812-ad95-4e08-8306-de1e85cc2c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:51.816716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d6c65812-ad95-4e08-8306-de1e85cc2c00 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:51.818065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d6c65812-ad95-4e08-8306-de1e85cc2c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:51.929796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1093/2185] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:51.941237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c65812-ad95-4e08-8306-de1e85cc2c00 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:51.943902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d6c65812-ad95-4e08-8306-de1e85cc2c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:51.960142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ab13df5fb304cd5b1ea67c5c749a0b5', 'request_id': 'req-92ab0440-b2ff-4948-90b7-49e94b38188c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:51.986377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ab13df5fb304cd5b1ea67c5c749a0b5', 'request_id': 'req-92ab0440-b2ff-4948-90b7-49e94b38188c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.018652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c65812-ad95-4e08-8306-de1e85cc2c00 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:52.083642 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ab13df5fb304cd5b1ea67c5c749a0b5', 'request_id': 'req-92ab0440-b2ff-4948-90b7-49e94b38188c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.112437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640b6f8-7842-42a2-96e8-192c8a8bacbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:52.116002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b6f8-7842-42a2-96e8-192c8a8bacbe service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:52.116915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b6f8-7842-42a2-96e8-192c8a8bacbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:52.120120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640b6f8-7842-42a2-96e8-192c8a8bacbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:52.130447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] The token cache is being invalidate because project 3ab13df5fb304cd5b1ea67c5c749a0b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:52.130575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.131519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.138143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1093/2186] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:51 2022] DELETE /identity/v3/projects/3ab13df5fb304cd5b1ea67c5c749a0b5 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:52.141613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:52.144266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:52.235386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640b6f8-7842-42a2-96e8-192c8a8bacbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:52.237681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640b6f8-7842-42a2-96e8-192c8a8bacbe service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:52.251825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1094/2187] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:52.302640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab0440-b2ff-4948-90b7-49e94b38188c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:52.312048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:52.312712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:52.312998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:52.313353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] PATH_INFO: `/v3/users/174d381b7c1f4b06a65bea4274440b58` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:52.314558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] RBAC: Authorizing `identity:delete_user(user_id=174d381b7c1f4b06a65bea4274440b58)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:52.317389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:52.471845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640b6f8-7842-42a2-96e8-192c8a8bacbe service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:52.474035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f640b6f8-7842-42a2-96e8-192c8a8bacbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:52.517642 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640b6f8-7842-42a2-96e8-192c8a8bacbe service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:52.559381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:52.563424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:52.563946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:52.564523 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] PATH_INFO: `/v3/users/a129445ddd474607852bf8c4f8cee59c` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:52.568543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] RBAC: Authorizing `identity:delete_user(user_id=a129445ddd474607852bf8c4f8cee59c)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:52.578791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:52.601027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:52.678377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174d381b7c1f4b06a65bea4274440b58'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.679869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174d381b7c1f4b06a65bea4274440b58'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.681296 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174d381b7c1f4b06a65bea4274440b58'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.696600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174d381b7c1f4b06a65bea4274440b58'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.718949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-9318dafa-2290-491c-8bb2-ce9907e2167e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174d381b7c1f4b06a65bea4274440b58'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.854948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1094/2188] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:52 2022] DELETE /identity/v3/users/174d381b7c1f4b06a65bea4274440b58 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:52.880598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a129445ddd474607852bf8c4f8cee59c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.930287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:52.964901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a129445ddd474607852bf8c4f8cee59c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:52.966098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a129445ddd474607852bf8c4f8cee59c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:53.020793 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a129445ddd474607852bf8c4f8cee59c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:53.021837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a129445ddd474607852bf8c4f8cee59c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:53.054554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1095/2189] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:52 2022] DELETE /identity/v3/users/a129445ddd474607852bf8c4f8cee59c => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:53.056397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f608c60-7ead-481e-afd7-5e33b8ea2130 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:53.056876 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f608c60-7ead-481e-afd7-5e33b8ea2130 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:53.057315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f608c60-7ead-481e-afd7-5e33b8ea2130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:53.122016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3f608c60-7ead-481e-afd7-5e33b8ea2130 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:53.126384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3f608c60-7ead-481e-afd7-5e33b8ea2130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:53.240503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1095/2190] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:53.257955 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:53.260418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:53.289293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233449c8-1ac0-45a3-b8d1-428a6f1972be admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:53.299150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f608c60-7ead-481e-afd7-5e33b8ea2130 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:53.302028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3f608c60-7ead-481e-afd7-5e33b8ea2130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:53.315169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:53.317630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:53.318461 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:53.320882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] PATH_INFO: `/v3/projects/6d4fa6545ff347e7b125d3921a8cc0dd` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:53.325379 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d4fa6545ff347e7b125d3921a8cc0dd)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:53.331952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:53.469473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f608c60-7ead-481e-afd7-5e33b8ea2130 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:53.476247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe497bb-1221-4d02-8cf0-1f8feea24f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:53.477087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe497bb-1221-4d02-8cf0-1f8feea24f74 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:53.477513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe497bb-1221-4d02-8cf0-1f8feea24f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:53.479304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe497bb-1221-4d02-8cf0-1f8feea24f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:53.492845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe497bb-1221-4d02-8cf0-1f8feea24f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:53.494682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe497bb-1221-4d02-8cf0-1f8feea24f74 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:53.509094 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1096/2191] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:53.583905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d4fa6545ff347e7b125d3921a8cc0dd', 'request_id': 'req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:53.590604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d4fa6545ff347e7b125d3921a8cc0dd', 'request_id': 'req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:53.724149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d4fa6545ff347e7b125d3921a8cc0dd', 'request_id': 'req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:53.735538 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe497bb-1221-4d02-8cf0-1f8feea24f74 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:53.738173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bbe497bb-1221-4d02-8cf0-1f8feea24f74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:53.756565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] The token cache is being invalidate because project 6d4fa6545ff347e7b125d3921a8cc0dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:53.756867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:53.760590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33770cdf-5746-4a12-bc58-dfd46f6ea8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:53.771204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe497bb-1221-4d02-8cf0-1f8feea24f74 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:53.773603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1096/2192] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:53 2022] DELETE /identity/v3/projects/6d4fa6545ff347e7b125d3921a8cc0dd => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:53.780203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea966e5-cb40-4609-b1aa-f437421be71a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:53.780513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea966e5-cb40-4609-b1aa-f437421be71a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:53.780798 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea966e5-cb40-4609-b1aa-f437421be71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:53.848091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-bea966e5-cb40-4609-b1aa-f437421be71a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:53.849810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-bea966e5-cb40-4609-b1aa-f437421be71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:53.907965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:53.908665 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:53.908950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:53.909304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] PATH_INFO: `/v3/projects/e69767a828744c4996558d3e3089f967` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:53.911509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e69767a828744c4996558d3e3089f967)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:53.913611 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:54.042875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1097/2193] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:54.051593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e632e-759c-4077-aab8-7d893597b5b4 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:54.052233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e632e-759c-4077-aab8-7d893597b5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:54.052776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e632e-759c-4077-aab8-7d893597b5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:54.091003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8d9e632e-759c-4077-aab8-7d893597b5b4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:54.093063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8d9e632e-759c-4077-aab8-7d893597b5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:54.097628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e69767a828744c4996558d3e3089f967', 'request_id': 'req-96b779ff-f7cc-489b-be52-2b4ce877f5d1'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.108990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e69767a828744c4996558d3e3089f967', 'request_id': 'req-96b779ff-f7cc-489b-be52-2b4ce877f5d1'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.226417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1098/2194] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:54.226417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e632e-759c-4077-aab8-7d893597b5b4 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:54.226417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8d9e632e-759c-4077-aab8-7d893597b5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:54.238814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e632e-759c-4077-aab8-7d893597b5b4 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:54.261517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:54.263050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:54.263050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:54.263050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] PATH_INFO: `/v3/users/fca7412ff12d4fcdbbcf996f67213864` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:54.265771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca7412ff12d4fcdbbcf996f67213864)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:54.268093 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:54.285745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e69767a828744c4996558d3e3089f967', 'request_id': 'req-96b779ff-f7cc-489b-be52-2b4ce877f5d1'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.315782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] The token cache is being invalidate because project e69767a828744c4996558d3e3089f967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:54.316940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.322836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.328166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1097/2195] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:53 2022] DELETE /identity/v3/projects/e69767a828744c4996558d3e3089f967 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:54.330025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:54.331148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:54.348166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b779ff-f7cc-489b-be52-2b4ce877f5d1 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:54.352521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f454a58-25af-4010-87c3-e8a61713039f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:54.353454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f454a58-25af-4010-87c3-e8a61713039f admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:54.353737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f454a58-25af-4010-87c3-e8a61713039f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:54.354010 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f454a58-25af-4010-87c3-e8a61713039f admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:54.354682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f454a58-25af-4010-87c3-e8a61713039f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:54.355002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f454a58-25af-4010-87c3-e8a61713039f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:54.356800 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f454a58-25af-4010-87c3-e8a61713039f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:54.376785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1098/2196] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:54.379918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f3aac-fb15-4b28-8616-4d4c1657e1b0 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:54.380597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f3aac-fb15-4b28-8616-4d4c1657e1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:54.381184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f3aac-fb15-4b28-8616-4d4c1657e1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:54.427011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ac0f3aac-fb15-4b28-8616-4d4c1657e1b0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:54.428500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ac0f3aac-fb15-4b28-8616-4d4c1657e1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:54.445812 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1099/2197] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:54.447591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f3aac-fb15-4b28-8616-4d4c1657e1b0 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:54.468182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f3aac-fb15-4b28-8616-4d4c1657e1b0 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:54.477144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86621edb-34d1-4523-b76a-e55c8d801b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:54.477828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86621edb-34d1-4523-b76a-e55c8d801b42 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:54.478317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86621edb-34d1-4523-b76a-e55c8d801b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:54.478597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86621edb-34d1-4523-b76a-e55c8d801b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:54.479866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86621edb-34d1-4523-b76a-e55c8d801b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:54.483441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86621edb-34d1-4523-b76a-e55c8d801b42 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:54.556000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca7412ff12d4fcdbbcf996f67213864'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.605719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1100/2198] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:54 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:54.606704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86621edb-34d1-4523-b76a-e55c8d801b42 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:54.609008 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-86621edb-34d1-4523-b76a-e55c8d801b42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:54.668932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca7412ff12d4fcdbbcf996f67213864'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.671214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca7412ff12d4fcdbbcf996f67213864'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.686730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86621edb-34d1-4523-b76a-e55c8d801b42 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:54.700978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca7412ff12d4fcdbbcf996f67213864'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.701782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8c74a379-afd7-4b9c-b577-b7fda1dd6fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca7412ff12d4fcdbbcf996f67213864'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.708844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:54.710436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:54.711996 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:54.712668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] PATH_INFO: `/v3/projects/d9adff9463f34ccab428829a3e3b32b0` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:54.718005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9adff9463f34ccab428829a3e3b32b0)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:54.721375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:54.733130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1099/2199] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:54 2022] DELETE /identity/v3/users/fca7412ff12d4fcdbbcf996f67213864 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:54.735101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344d4a0-8591-494e-a0ef-ccf58b5f71b7 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:54.742738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344d4a0-8591-494e-a0ef-ccf58b5f71b7 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:54.742738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344d4a0-8591-494e-a0ef-ccf58b5f71b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:54.769008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b344d4a0-8591-494e-a0ef-ccf58b5f71b7 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:54.771067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b344d4a0-8591-494e-a0ef-ccf58b5f71b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:54.876617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9adff9463f34ccab428829a3e3b32b0', 'request_id': 'req-91116ee9-41fb-487c-870c-4c7da344f814'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.886865 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1100/2200] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:54.888128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344d4a0-8591-494e-a0ef-ccf58b5f71b7 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:54.892382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b344d4a0-8591-494e-a0ef-ccf58b5f71b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:54.903036 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:54.913021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9adff9463f34ccab428829a3e3b32b0', 'request_id': 'req-91116ee9-41fb-487c-870c-4c7da344f814'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:54.924820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344d4a0-8591-494e-a0ef-ccf58b5f71b7 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:54.993560 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9adff9463f34ccab428829a3e3b32b0', 'request_id': 'req-91116ee9-41fb-487c-870c-4c7da344f814'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:55.017853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] The token cache is being invalidate because project d9adff9463f34ccab428829a3e3b32b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:55.018427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:55.020432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91116ee9-41fb-487c-870c-4c7da344f814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:55.034746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6b96e-02f4-48ba-97dd-a7c4dc3ea1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:55.034746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6b96e-02f4-48ba-97dd-a7c4dc3ea1ef admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:55.034746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6b96e-02f4-48ba-97dd-a7c4dc3ea1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:55.034746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6b96e-02f4-48ba-97dd-a7c4dc3ea1ef admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:55.035104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6b96e-02f4-48ba-97dd-a7c4dc3ea1ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:55.035104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6b96e-02f4-48ba-97dd-a7c4dc3ea1ef admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:55.035272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1101/2201] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:54 2022] DELETE /identity/v3/projects/d9adff9463f34ccab428829a3e3b32b0 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:55.035272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c6d40-6612-4afe-9ddd-3d4868dd37b4 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:55.035272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c6d40-6612-4afe-9ddd-3d4868dd37b4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:55.035272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c6d40-6612-4afe-9ddd-3d4868dd37b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:55.068404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-376c6d40-6612-4afe-9ddd-3d4868dd37b4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:55.070668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-376c6d40-6612-4afe-9ddd-3d4868dd37b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:55.085155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1102/2202] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:55.086638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c6d40-6612-4afe-9ddd-3d4868dd37b4 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:55.098576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-376c6d40-6612-4afe-9ddd-3d4868dd37b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:55.119750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c6d40-6612-4afe-9ddd-3d4868dd37b4 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:55.128839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfc9d3-e1e4-49df-9871-6e2cc4b0aa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:55.129823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc9d3-e1e4-49df-9871-6e2cc4b0aa0f admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:55.130322 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc9d3-e1e4-49df-9871-6e2cc4b0aa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:55.130862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfc9d3-e1e4-49df-9871-6e2cc4b0aa0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:55.132146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfc9d3-e1e4-49df-9871-6e2cc4b0aa0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:55.132681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfc9d3-e1e4-49df-9871-6e2cc4b0aa0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:55.138008 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfc9d3-e1e4-49df-9871-6e2cc4b0aa0f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:55.149767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1101/2203] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:54 2022] POST /identity/v3/users => generated 292 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:55.150031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6b96e-02f4-48ba-97dd-a7c4dc3ea1ef admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:55.153196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8ae6b96e-02f4-48ba-97dd-a7c4dc3ea1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:55.155416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1103/2204] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:31:55.156406 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfc9d3-e1e4-49df-9871-6e2cc4b0aa0f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:55.162159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c4dfc9d3-e1e4-49df-9871-6e2cc4b0aa0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:55.175597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6b96e-02f4-48ba-97dd-a7c4dc3ea1ef admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:55.184163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:55.186341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:55.186810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:55.187218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] PATH_INFO: `/v3/users/15031c9616cc437ebf69f41d9b325a94` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:55.189120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15031c9616cc437ebf69f41d9b325a94)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:55.192896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:55.249249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:55.280771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfc9d3-e1e4-49df-9871-6e2cc4b0aa0f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:55.289361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104606cb-0707-496b-9098-f6b9342b542d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:55.290459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104606cb-0707-496b-9098-f6b9342b542d service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:55.290955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104606cb-0707-496b-9098-f6b9342b542d service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:55.291530 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104606cb-0707-496b-9098-f6b9342b542d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:55.312371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104606cb-0707-496b-9098-f6b9342b542d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:55.315312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104606cb-0707-496b-9098-f6b9342b542d service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:55.335612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1104/2205] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:55.337057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104606cb-0707-496b-9098-f6b9342b542d service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:55.339304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-104606cb-0707-496b-9098-f6b9342b542d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:55.364481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104606cb-0707-496b-9098-f6b9342b542d service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:55.375477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d82c95-0888-4344-94e2-fe502627548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:55.376599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d82c95-0888-4344-94e2-fe502627548a admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:55.377188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d82c95-0888-4344-94e2-fe502627548a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:55.377712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d82c95-0888-4344-94e2-fe502627548a admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:55.379313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d82c95-0888-4344-94e2-fe502627548a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:55.387475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d82c95-0888-4344-94e2-fe502627548a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:55.411610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1105/2206] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:55.412825 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d82c95-0888-4344-94e2-fe502627548a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:55.432507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15031c9616cc437ebf69f41d9b325a94'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:55.436741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d82c95-0888-4344-94e2-fe502627548a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:55.447121 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b097233-d886-4f66-bfb4-127c6649c1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:55.448059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b097233-d886-4f66-bfb4-127c6649c1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:55.448526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b097233-d886-4f66-bfb4-127c6649c1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:55.448958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b097233-d886-4f66-bfb4-127c6649c1a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:55.449807 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b097233-d886-4f66-bfb4-127c6649c1a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:55.452081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b097233-d886-4f66-bfb4-127c6649c1a9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:55.577263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15031c9616cc437ebf69f41d9b325a94'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:55.578864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15031c9616cc437ebf69f41d9b325a94'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:55.593959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1106/2207] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:55 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:55.596063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b097233-d886-4f66-bfb4-127c6649c1a9 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:55.602547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3b097233-d886-4f66-bfb4-127c6649c1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:55.615009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15031c9616cc437ebf69f41d9b325a94'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:55.616224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15031c9616cc437ebf69f41d9b325a94'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:55.645794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1102/2208] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:55 2022] DELETE /identity/v3/users/15031c9616cc437ebf69f41d9b325a94 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:55.647062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:55.651784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:55.768428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b097233-d886-4f66-bfb4-127c6649c1a9 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:55.814988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd90426-61fc-4c2e-8119-6e04d0122614 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:55.826484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4cab2-a843-4b4a-83f9-5767a7d312f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:55.829597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4cab2-a843-4b4a-83f9-5767a7d312f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:55.829597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4cab2-a843-4b4a-83f9-5767a7d312f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:55.829597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b4cab2-a843-4b4a-83f9-5767a7d312f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:55.829597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4cab2-a843-4b4a-83f9-5767a7d312f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:55.833238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b4cab2-a843-4b4a-83f9-5767a7d312f8 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:55.900781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df039a51-7341-4aee-8d90-f1054ea79b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:55.903854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df039a51-7341-4aee-8d90-f1054ea79b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:55.903854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df039a51-7341-4aee-8d90-f1054ea79b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:55.903854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df039a51-7341-4aee-8d90-f1054ea79b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:55.904895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df039a51-7341-4aee-8d90-f1054ea79b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:55.908168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df039a51-7341-4aee-8d90-f1054ea79b25 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:55.937110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1107/2209] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:55.943065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df039a51-7341-4aee-8d90-f1054ea79b25 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:55.968032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df039a51-7341-4aee-8d90-f1054ea79b25 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:55.982161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46771b-dcf1-41e9-9488-8243f817c0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:55.983761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46771b-dcf1-41e9-9488-8243f817c0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:55.984242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46771b-dcf1-41e9-9488-8243f817c0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:55.984927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46771b-dcf1-41e9-9488-8243f817c0cb admin admin] PATH_INFO: `/v3/projects/95935852764e413c8bbd23c16ddf13a5/users/c8982a7c52314c5a82d80a03d9a833d6/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:56.012593 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46771b-dcf1-41e9-9488-8243f817c0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=95935852764e413c8bbd23c16ddf13a5, user_id=c8982a7c52314c5a82d80a03d9a833d6, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:56.017809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46771b-dcf1-41e9-9488-8243f817c0cb admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:56.064073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1103/2210] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:55 2022] POST /identity/v3/users => generated 293 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:56.085081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4cab2-a843-4b4a-83f9-5767a7d312f8 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:56.116025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b4cab2-a843-4b4a-83f9-5767a7d312f8 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:56.155254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e3457-0199-4487-862e-ac9d0cde5e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:56.157763 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e3457-0199-4487-862e-ac9d0cde5e2e admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:56.158226 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e3457-0199-4487-862e-ac9d0cde5e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:56.158580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e3457-0199-4487-862e-ac9d0cde5e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:56.160525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e3457-0199-4487-862e-ac9d0cde5e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:56.166043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94e3457-0199-4487-862e-ac9d0cde5e2e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:56.179640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1108/2211] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:55 2022] PUT /identity/v3/projects/95935852764e413c8bbd23c16ddf13a5/users/c8982a7c52314c5a82d80a03d9a833d6/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:31:56.187263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e173084-51ed-4005-bfec-b130f47f89de None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:56.187861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e173084-51ed-4005-bfec-b130f47f89de None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:56.188575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e173084-51ed-4005-bfec-b130f47f89de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:56.195454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1104/2212] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:56.197208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e3457-0199-4487-862e-ac9d0cde5e2e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:56.216091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e3457-0199-4487-862e-ac9d0cde5e2e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:56.245125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6e173084-51ed-4005-bfec-b130f47f89de None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:56.246616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6e173084-51ed-4005-bfec-b130f47f89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:56.350408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1109/2213] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:56.367722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e173084-51ed-4005-bfec-b130f47f89de None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:56.370446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6e173084-51ed-4005-bfec-b130f47f89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:56.463312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0efb47-36b9-4d01-8ee5-8d08179f8521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:56.464608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0efb47-36b9-4d01-8ee5-8d08179f8521 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:56.465180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0efb47-36b9-4d01-8ee5-8d08179f8521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:56.465775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0efb47-36b9-4d01-8ee5-8d08179f8521 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:56.467988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0efb47-36b9-4d01-8ee5-8d08179f8521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:56.471528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0efb47-36b9-4d01-8ee5-8d08179f8521 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:56.491697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1105/2214] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:56 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:56.493285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0efb47-36b9-4d01-8ee5-8d08179f8521 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:56.496969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ca0efb47-36b9-4d01-8ee5-8d08179f8521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:56.521859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0efb47-36b9-4d01-8ee5-8d08179f8521 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:56.530854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:56.531905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:56.532257 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:56.532571 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] PATH_INFO: `/v3/projects/4a56af79aca1481ea87944a95624b3d1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:56.534484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a56af79aca1481ea87944a95624b3d1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:56.536736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e173084-51ed-4005-bfec-b130f47f89de None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:56.538184 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:56.543044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7840-657d-4c47-812c-cef897739516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:56.544211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7840-657d-4c47-812c-cef897739516 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:56.544567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7840-657d-4c47-812c-cef897739516 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:56.545412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae7840-657d-4c47-812c-cef897739516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:56.558480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae7840-657d-4c47-812c-cef897739516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:56.561316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae7840-657d-4c47-812c-cef897739516 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:56.577225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1110/2215] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:31:56.579644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7840-657d-4c47-812c-cef897739516 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:56.611759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae7840-657d-4c47-812c-cef897739516 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:56.616357 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffcd71-3910-4d84-a1d2-b6385f1350be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:56.617686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffcd71-3910-4d84-a1d2-b6385f1350be admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:56.618269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffcd71-3910-4d84-a1d2-b6385f1350be admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:56.618863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffcd71-3910-4d84-a1d2-b6385f1350be admin admin] PATH_INFO: `/v3/projects/9903ae33db5a4002afc782eb4fba7612/users/67c97168b0484f70aeaf558b84b9f99c/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:56.651384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffcd71-3910-4d84-a1d2-b6385f1350be admin admin] RBAC: Authorizing `identity:create_grant(project_id=9903ae33db5a4002afc782eb4fba7612, user_id=67c97168b0484f70aeaf558b84b9f99c, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:56.662758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffcd71-3910-4d84-a1d2-b6385f1350be admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:56.795600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1111/2216] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:56 2022] PUT /identity/v3/projects/9903ae33db5a4002afc782eb4fba7612/users/67c97168b0484f70aeaf558b84b9f99c/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:56.795600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffcd71-3910-4d84-a1d2-b6385f1350be admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:56.800587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f8ffcd71-3910-4d84-a1d2-b6385f1350be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:56.821285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a56af79aca1481ea87944a95624b3d1', 'request_id': 'req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:56.832918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a56af79aca1481ea87944a95624b3d1', 'request_id': 'req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:56.918793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffcd71-3910-4d84-a1d2-b6385f1350be admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:56.937368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a56af79aca1481ea87944a95624b3d1', 'request_id': 'req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:56.971313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] The token cache is being invalidate because project 4a56af79aca1481ea87944a95624b3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:56.972358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:56.973932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:56.975164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-33e9fbd6-9c3b-4816-a2ae-3d74a11e9df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:56.979069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1106/2217] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:56 2022] DELETE /identity/v3/projects/4a56af79aca1481ea87944a95624b3d1 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:56.983003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1060403-c9eb-4a07-8db4-1468a5e877a9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:56.983758 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1060403-c9eb-4a07-8db4-1468a5e877a9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:56.984530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1060403-c9eb-4a07-8db4-1468a5e877a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:57.088042 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b1060403-c9eb-4a07-8db4-1468a5e877a9 None None] MFA Rules not processed for user `67c97168b0484f70aeaf558b84b9f99c`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:57.089608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b1060403-c9eb-4a07-8db4-1468a5e877a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:57.111433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:57.115197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:57.116042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:57.116702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] PATH_INFO: `/v3/projects/ebc65d4e3d42431db5d52483451a0ae0` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:57.126267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc65d4e3d42431db5d52483451a0ae0)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:57.129792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:57.226390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1107/2218] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:57.226390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1060403-c9eb-4a07-8db4-1468a5e877a9 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:57.226680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b1060403-c9eb-4a07-8db4-1468a5e877a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:57.245599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1060403-c9eb-4a07-8db4-1468a5e877a9 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:57.251885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:57.252610 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:57.252928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:57.253223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] PATH_INFO: `/v3/users/7532a489d0b74122a5ff3069fd9997a0` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:57.255234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7532a489d0b74122a5ff3069fd9997a0)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:57.257779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:57.337507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc65d4e3d42431db5d52483451a0ae0', 'request_id': 'req-20e37177-ac8d-41f2-abee-0e8b272a0bd8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:57.348283 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc65d4e3d42431db5d52483451a0ae0', 'request_id': 'req-20e37177-ac8d-41f2-abee-0e8b272a0bd8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:57.490387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc65d4e3d42431db5d52483451a0ae0', 'request_id': 'req-20e37177-ac8d-41f2-abee-0e8b272a0bd8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:57.514084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] The token cache is being invalidate because project ebc65d4e3d42431db5d52483451a0ae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:57.514404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:57.515346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:57.517489 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1112/2219] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:56 2022] DELETE /identity/v3/projects/ebc65d4e3d42431db5d52483451a0ae0 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:57.519650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:57.519809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:57.615711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:57.615968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e37177-ac8d-41f2-abee-0e8b272a0bd8 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:57.621919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2e624-43df-4d5a-85cf-d48714bce995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:57.622619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2e624-43df-4d5a-85cf-d48714bce995 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:57.622906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2e624-43df-4d5a-85cf-d48714bce995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:57.623179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2e624-43df-4d5a-85cf-d48714bce995 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:57.623893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2e624-43df-4d5a-85cf-d48714bce995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:57.626084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2e624-43df-4d5a-85cf-d48714bce995 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:57.665743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7532a489d0b74122a5ff3069fd9997a0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:57.779626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7532a489d0b74122a5ff3069fd9997a0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:57.780419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7532a489d0b74122a5ff3069fd9997a0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:57.802660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7532a489d0b74122a5ff3069fd9997a0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:57.803750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6006ce6f-afdb-4add-9d75-0ac947c03de8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7532a489d0b74122a5ff3069fd9997a0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:57.834797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1108/2220] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:57 2022] DELETE /identity/v3/users/7532a489d0b74122a5ff3069fd9997a0 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:57.836587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670814-662c-41c0-bcfe-cf4b1fd917b3 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:57.836878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670814-662c-41c0-bcfe-cf4b1fd917b3 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:57.837308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92670814-662c-41c0-bcfe-cf4b1fd917b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:57.860367 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1113/2221] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:57 2022] POST /identity/v3/projects => generated 387 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:57.861028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-92670814-662c-41c0-bcfe-cf4b1fd917b3 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:57.861511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2e624-43df-4d5a-85cf-d48714bce995 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:57.862669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-92670814-662c-41c0-bcfe-cf4b1fd917b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:57.865519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b2c2e624-43df-4d5a-85cf-d48714bce995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:57.973694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2e624-43df-4d5a-85cf-d48714bce995 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:57.986875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1109/2222] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:57.988897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c5264-3350-4a95-b927-4a46f1a19978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:57.992288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5264-3350-4a95-b927-4a46f1a19978 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:57.993010 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5264-3350-4a95-b927-4a46f1a19978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:57.993255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5264-3350-4a95-b927-4a46f1a19978 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:57.994542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51192f5f-38bb-42b8-aea4-8132034b34b5 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:57.994703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51192f5f-38bb-42b8-aea4-8132034b34b5 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:57.995038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51192f5f-38bb-42b8-aea4-8132034b34b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:57.999969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c5264-3350-4a95-b927-4a46f1a19978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.002059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81c5264-3350-4a95-b927-4a46f1a19978 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.020776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-51192f5f-38bb-42b8-aea4-8132034b34b5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:58.022167 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-51192f5f-38bb-42b8-aea4-8132034b34b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:58.031421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1110/2223] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:31:58.036285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51192f5f-38bb-42b8-aea4-8132034b34b5 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.046525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51192f5f-38bb-42b8-aea4-8132034b34b5 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.052000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc35740-7eea-4f79-89f2-f377cd82b42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.052624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35740-7eea-4f79-89f2-f377cd82b42f admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.053020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35740-7eea-4f79-89f2-f377cd82b42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.053576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35740-7eea-4f79-89f2-f377cd82b42f admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.054238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35740-7eea-4f79-89f2-f377cd82b42f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:31:58.054544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35740-7eea-4f79-89f2-f377cd82b42f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.056196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35740-7eea-4f79-89f2-f377cd82b42f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.078092 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1111/2224] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:31:58.082279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc35740-7eea-4f79-89f2-f377cd82b42f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.092098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc35740-7eea-4f79-89f2-f377cd82b42f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.099350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8b053-5cdc-4346-8f4d-4929f8e0ab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.100614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8b053-5cdc-4346-8f4d-4929f8e0ab72 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.101050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8b053-5cdc-4346-8f4d-4929f8e0ab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.101364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8b053-5cdc-4346-8f4d-4929f8e0ab72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.102143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8b053-5cdc-4346-8f4d-4929f8e0ab72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.104834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8b053-5cdc-4346-8f4d-4929f8e0ab72 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.144572 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1114/2225] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:57 2022] POST /identity/v3/users => generated 300 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:58.149017 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c5264-3350-4a95-b927-4a46f1a19978 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.168054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81c5264-3350-4a95-b927-4a46f1a19978 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.175831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54884888-5200-4f4c-b8f6-ac9462e5c91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.176860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54884888-5200-4f4c-b8f6-ac9462e5c91d admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.177283 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54884888-5200-4f4c-b8f6-ac9462e5c91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.177791 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54884888-5200-4f4c-b8f6-ac9462e5c91d admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.179185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54884888-5200-4f4c-b8f6-ac9462e5c91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.182218 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54884888-5200-4f4c-b8f6-ac9462e5c91d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.197472 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1115/2226] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:31:58.202939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54884888-5200-4f4c-b8f6-ac9462e5c91d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.240463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1112/2227] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:58 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:58.246205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8b053-5cdc-4346-8f4d-4929f8e0ab72 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.249731 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54884888-5200-4f4c-b8f6-ac9462e5c91d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.393067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8b053-5cdc-4346-8f4d-4929f8e0ab72 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.407033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d99de1-c663-4b6d-bce6-317c380bf253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.408202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d99de1-c663-4b6d-bce6-317c380bf253 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.408706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d99de1-c663-4b6d-bce6-317c380bf253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.409195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d99de1-c663-4b6d-bce6-317c380bf253 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.410383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d99de1-c663-4b6d-bce6-317c380bf253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.416258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d99de1-c663-4b6d-bce6-317c380bf253 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.429138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83057562-d844-40e2-9ce8-7cf00b49d3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.429138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83057562-d844-40e2-9ce8-7cf00b49d3f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.429138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83057562-d844-40e2-9ce8-7cf00b49d3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.429138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83057562-d844-40e2-9ce8-7cf00b49d3f6 admin admin] PATH_INFO: `/v3/projects/26ed1108cdc04b768050eb65b0159659/users/6c4879d538d14286a0eddbb9f4707593/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.446325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83057562-d844-40e2-9ce8-7cf00b49d3f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ed1108cdc04b768050eb65b0159659, user_id=6c4879d538d14286a0eddbb9f4707593, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.450331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83057562-d844-40e2-9ce8-7cf00b49d3f6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.556135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1113/2228] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:58 2022] POST /identity/v3/users => generated 292 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:58.557784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc6b29-6905-49c2-ae7f-459ab1eb7968 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.557784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc6b29-6905-49c2-ae7f-459ab1eb7968 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.557784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cc6b29-6905-49c2-ae7f-459ab1eb7968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.585754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1116/2229] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:58 2022] PUT /identity/v3/projects/26ed1108cdc04b768050eb65b0159659/users/6c4879d538d14286a0eddbb9f4707593/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:58.588287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83057562-d844-40e2-9ce8-7cf00b49d3f6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.613628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-66cc6b29-6905-49c2-ae7f-459ab1eb7968 None None] MFA Rules not processed for user `c8982a7c52314c5a82d80a03d9a833d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:58.614882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-66cc6b29-6905-49c2-ae7f-459ab1eb7968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:58.669711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:58.694264 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83057562-d844-40e2-9ce8-7cf00b49d3f6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.700321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66fbac-a2a7-43b7-8713-144cc5d7099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.701040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66fbac-a2a7-43b7-8713-144cc5d7099e admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.701998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66fbac-a2a7-43b7-8713-144cc5d7099e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.702351 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66fbac-a2a7-43b7-8713-144cc5d7099e admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.702960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66fbac-a2a7-43b7-8713-144cc5d7099e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.705380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66fbac-a2a7-43b7-8713-144cc5d7099e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.707905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1114/2230] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:58.709050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc6b29-6905-49c2-ae7f-459ab1eb7968 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.727086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1117/2231] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:58.727086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66fbac-a2a7-43b7-8713-144cc5d7099e admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.729136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ab66fbac-a2a7-43b7-8713-144cc5d7099e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:58.742146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cc6b29-6905-49c2-ae7f-459ab1eb7968 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.754721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777854ac-7e99-4481-b403-7d66b4c8358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.755366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777854ac-7e99-4481-b403-7d66b4c8358d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.755680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777854ac-7e99-4481-b403-7d66b4c8358d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.755956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777854ac-7e99-4481-b403-7d66b4c8358d admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.756628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777854ac-7e99-4481-b403-7d66b4c8358d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.764595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777854ac-7e99-4481-b403-7d66b4c8358d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.790733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1115/2232] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:58.793271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777854ac-7e99-4481-b403-7d66b4c8358d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.797648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-777854ac-7e99-4481-b403-7d66b4c8358d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:58.812856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777854ac-7e99-4481-b403-7d66b4c8358d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.819474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.820276 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.820625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.820976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] PATH_INFO: `/v3/projects/cf2cdcc5b125423c8a3dedf5a4cf9e32` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.822367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf2cdcc5b125423c8a3dedf5a4cf9e32)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.829251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.836494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66fbac-a2a7-43b7-8713-144cc5d7099e admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.840842 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fab734-329a-4e32-b6cd-6d5ea18d7221 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.841582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fab734-329a-4e32-b6cd-6d5ea18d7221 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.842116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fab734-329a-4e32-b6cd-6d5ea18d7221 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.842394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fab734-329a-4e32-b6cd-6d5ea18d7221 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.853782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fab734-329a-4e32-b6cd-6d5ea18d7221 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.855831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fab734-329a-4e32-b6cd-6d5ea18d7221 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.866415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1118/2233] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:58.867304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fab734-329a-4e32-b6cd-6d5ea18d7221 service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.881075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fab734-329a-4e32-b6cd-6d5ea18d7221 service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.887183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b5a41-93cc-477d-b0f4-472eb29dbb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.887953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b5a41-93cc-477d-b0f4-472eb29dbb18 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.888410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b5a41-93cc-477d-b0f4-472eb29dbb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.888706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b5a41-93cc-477d-b0f4-472eb29dbb18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.889411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b5a41-93cc-477d-b0f4-472eb29dbb18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.891071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b5a41-93cc-477d-b0f4-472eb29dbb18 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.905645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1119/2234] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:31:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:31:58.905645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b5a41-93cc-477d-b0f4-472eb29dbb18 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.914055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b5a41-93cc-477d-b0f4-472eb29dbb18 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:58.921471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741025d-d14a-4fea-af0f-c7aac1286476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:58.922188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741025d-d14a-4fea-af0f-c7aac1286476 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:58.922469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741025d-d14a-4fea-af0f-c7aac1286476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:58.922752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741025d-d14a-4fea-af0f-c7aac1286476 admin admin] PATH_INFO: `/v3/projects/26ed1108cdc04b768050eb65b0159659/users/6c4879d538d14286a0eddbb9f4707593/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:58.924963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741025d-d14a-4fea-af0f-c7aac1286476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ed1108cdc04b768050eb65b0159659, user_id=6c4879d538d14286a0eddbb9f4707593, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:58.928070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741025d-d14a-4fea-af0f-c7aac1286476 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:58.943118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1120/2235] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:58 2022] PUT /identity/v3/projects/26ed1108cdc04b768050eb65b0159659/users/6c4879d538d14286a0eddbb9f4707593/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:58.944058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741025d-d14a-4fea-af0f-c7aac1286476 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:58.962354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf2cdcc5b125423c8a3dedf5a4cf9e32', 'request_id': 'req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:58.973947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf2cdcc5b125423c8a3dedf5a4cf9e32', 'request_id': 'req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:59.069315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741025d-d14a-4fea-af0f-c7aac1286476 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:59.141657 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf2cdcc5b125423c8a3dedf5a4cf9e32', 'request_id': 'req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:59.156922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b64692-4270-45b6-b461-717a90044b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:59.157978 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b64692-4270-45b6-b461-717a90044b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:59.158161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b64692-4270-45b6-b461-717a90044b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:59.158403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b64692-4270-45b6-b461-717a90044b8f admin admin] PATH_INFO: `/v3/projects/de0dcb7a70894391a6c06acb0a2e0eae/users/eec4e0e64cd3466db7295a98921df777/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:59.164075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] The token cache is being invalidate because project cf2cdcc5b125423c8a3dedf5a4cf9e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:31:59.164478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:59.165353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-22db1f9d-ef37-4a59-b895-cd835ee5dbf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:59.171385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1116/2236] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:31:58 2022] DELETE /identity/v3/projects/cf2cdcc5b125423c8a3dedf5a4cf9e32 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:59.171385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4c77-1906-406f-bb08-d1cab6d6fbcb None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:59.171385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4c77-1906-406f-bb08-d1cab6d6fbcb None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:59.171385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4c77-1906-406f-bb08-d1cab6d6fbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:59.184018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b64692-4270-45b6-b461-717a90044b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=de0dcb7a70894391a6c06acb0a2e0eae, user_id=eec4e0e64cd3466db7295a98921df777, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:59.189666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b64692-4270-45b6-b461-717a90044b8f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:59.194687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:31:59.220608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-88eb4c77-1906-406f-bb08-d1cab6d6fbcb None None] MFA Rules not processed for user `6c4879d538d14286a0eddbb9f4707593`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:59.221893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-88eb4c77-1906-406f-bb08-d1cab6d6fbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:59.313478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1117/2237] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:59.314413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4c77-1906-406f-bb08-d1cab6d6fbcb None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:59.315881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-88eb4c77-1906-406f-bb08-d1cab6d6fbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:59.330893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4c77-1906-406f-bb08-d1cab6d6fbcb None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:59.335949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:59.336600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:59.336884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:59.337166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] PATH_INFO: `/v3/users/bcb5e00d77924bc5afe7dc77cefee8ee` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:59.338358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb5e00d77924bc5afe7dc77cefee8ee)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:59.340498 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:59.382894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1121/2238] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:31:58 2022] PUT /identity/v3/projects/de0dcb7a70894391a6c06acb0a2e0eae/users/eec4e0e64cd3466db7295a98921df777/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:59.384175 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b64692-4270-45b6-b461-717a90044b8f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:59.389521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-71b64692-4270-45b6-b461-717a90044b8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:59.467440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b64692-4270-45b6-b461-717a90044b8f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:59.471462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c28eda-8c0b-410d-839d-10a564f3372b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:59.472130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c28eda-8c0b-410d-839d-10a564f3372b service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:59.472415 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c28eda-8c0b-410d-839d-10a564f3372b service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:59.472710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c28eda-8c0b-410d-839d-10a564f3372b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:59.548886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c28eda-8c0b-410d-839d-10a564f3372b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:59.550790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c28eda-8c0b-410d-839d-10a564f3372b service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:59.565984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1122/2239] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:31:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:31:59.568341 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3e6eb-ab50-49d9-8b24-0b97e1434220 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:59.568801 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3e6eb-ab50-49d9-8b24-0b97e1434220 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:59.569256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3e6eb-ab50-49d9-8b24-0b97e1434220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:59.612204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b6b3e6eb-ab50-49d9-8b24-0b97e1434220 None None] MFA Rules not processed for user `eec4e0e64cd3466db7295a98921df777`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:31:59.613844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b6b3e6eb-ab50-49d9-8b24-0b97e1434220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:59.688070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1123/2240] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:31:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:31:59.694687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3e6eb-ab50-49d9-8b24-0b97e1434220 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:59.696501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b6b3e6eb-ab50-49d9-8b24-0b97e1434220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:31:59.701827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb5e00d77924bc5afe7dc77cefee8ee'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:59.795152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3e6eb-ab50-49d9-8b24-0b97e1434220 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:31:59.800658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ec61f-227e-4605-9e0e-42013c69be5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:31:59.801558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ec61f-227e-4605-9e0e-42013c69be5c admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:31:59.801900 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ec61f-227e-4605-9e0e-42013c69be5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:31:59.802214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238ec61f-227e-4605-9e0e-42013c69be5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:31:59.802868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ec61f-227e-4605-9e0e-42013c69be5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:31:59.804867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238ec61f-227e-4605-9e0e-42013c69be5c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:31:59.807106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb5e00d77924bc5afe7dc77cefee8ee'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:59.808385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb5e00d77924bc5afe7dc77cefee8ee'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:59.832479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb5e00d77924bc5afe7dc77cefee8ee'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:59.833907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb5e00d77924bc5afe7dc77cefee8ee'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:31:59.849479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1118/2241] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:31:59 2022] DELETE /identity/v3/users/bcb5e00d77924bc5afe7dc77cefee8ee => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:31:59.962886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1124/2242] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:31:59 2022] POST /identity/v3/projects => generated 381 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:31:59.964802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:31:59.967969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:00.110768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb98d9-3b0b-436f-92fb-ab637f57c3d6 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:00.181939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ed20a-dd68-4c5b-a978-3a0dd149538d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:00.182851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ed20a-dd68-4c5b-a978-3a0dd149538d admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:00.183271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ed20a-dd68-4c5b-a978-3a0dd149538d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:00.183721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ed20a-dd68-4c5b-a978-3a0dd149538d admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:00.184749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ed20a-dd68-4c5b-a978-3a0dd149538d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:00.188795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ed20a-dd68-4c5b-a978-3a0dd149538d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:00.370914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1119/2243] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:31:59 2022] POST /identity/v3/users => generated 297 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:32:00.378272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ec61f-227e-4605-9e0e-42013c69be5c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:00.390644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238ec61f-227e-4605-9e0e-42013c69be5c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:00.396187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55279-bb03-40ef-aff6-d8d10551fc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:00.396810 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55279-bb03-40ef-aff6-d8d10551fc68 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:00.397093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55279-bb03-40ef-aff6-d8d10551fc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:00.397365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55279-bb03-40ef-aff6-d8d10551fc68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:00.398049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55279-bb03-40ef-aff6-d8d10551fc68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:00.399697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55279-bb03-40ef-aff6-d8d10551fc68 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:00.419945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1125/2244] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:32:00.432230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ed20a-dd68-4c5b-a978-3a0dd149538d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:00.451064 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ed20a-dd68-4c5b-a978-3a0dd149538d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:00.458852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a474584-fd4a-4a84-bb2b-ab2b42ed9821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:00.460166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a474584-fd4a-4a84-bb2b-ab2b42ed9821 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:00.460737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a474584-fd4a-4a84-bb2b-ab2b42ed9821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:00.461022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a474584-fd4a-4a84-bb2b-ab2b42ed9821 admin admin] PATH_INFO: `/v3/projects/a5b5b831866440918b5edaf2a09bc8be/users/f43b25af4f724e819af599b3bb332e8e/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:00.488833 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a474584-fd4a-4a84-bb2b-ab2b42ed9821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b5b831866440918b5edaf2a09bc8be, user_id=f43b25af4f724e819af599b3bb332e8e, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:00.491428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a474584-fd4a-4a84-bb2b-ab2b42ed9821 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:00.570391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1120/2245] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:00 2022] PUT /identity/v3/projects/a5b5b831866440918b5edaf2a09bc8be/users/f43b25af4f724e819af599b3bb332e8e/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:00.577755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55279-bb03-40ef-aff6-d8d10551fc68 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:00.595690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55279-bb03-40ef-aff6-d8d10551fc68 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:00.676387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9199a-a304-4aa1-8165-28972f7080c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:00.677198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9199a-a304-4aa1-8165-28972f7080c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:00.677619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9199a-a304-4aa1-8165-28972f7080c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:00.678043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9199a-a304-4aa1-8165-28972f7080c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:00.678794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9199a-a304-4aa1-8165-28972f7080c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:00.680609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc9199a-a304-4aa1-8165-28972f7080c4 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:00.692599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1126/2246] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:00 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:32:00.697899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a474584-fd4a-4a84-bb2b-ab2b42ed9821 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:00.716969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a474584-fd4a-4a84-bb2b-ab2b42ed9821 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:00.721629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9199a-a304-4aa1-8165-28972f7080c4 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:00.724291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-bdc9199a-a304-4aa1-8165-28972f7080c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:00.726361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557216f-98c5-4f49-9389-51d5c49f103f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:00.727248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557216f-98c5-4f49-9389-51d5c49f103f admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:00.727838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557216f-98c5-4f49-9389-51d5c49f103f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:00.732651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557216f-98c5-4f49-9389-51d5c49f103f admin admin] PATH_INFO: `/v3/projects/a5b5b831866440918b5edaf2a09bc8be/users/f43b25af4f724e819af599b3bb332e8e/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:00.732651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e557216f-98c5-4f49-9389-51d5c49f103f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b5b831866440918b5edaf2a09bc8be, user_id=f43b25af4f724e819af599b3bb332e8e, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:00.734718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e557216f-98c5-4f49-9389-51d5c49f103f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:00.746113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc9199a-a304-4aa1-8165-28972f7080c4 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:00.753792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1121/2247] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:00 2022] PUT /identity/v3/projects/a5b5b831866440918b5edaf2a09bc8be/users/f43b25af4f724e819af599b3bb332e8e/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:00.771842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecac595-0018-4497-afc9-84412d9139d6 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:00.778105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecac595-0018-4497-afc9-84412d9139d6 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:00.778105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecac595-0018-4497-afc9-84412d9139d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:00.820053 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-cecac595-0018-4497-afc9-84412d9139d6 None None] MFA Rules not processed for user `f43b25af4f724e819af599b3bb332e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:00.822113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cecac595-0018-4497-afc9-84412d9139d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:00.890496 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:00.891913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:00.892444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:00.893074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] PATH_INFO: `/v3/projects/46383422ee4d44fdab36c49f904b8440` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:00.895893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] RBAC: Authorizing `identity:delete_project(project_id=46383422ee4d44fdab36c49f904b8440)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:00.900098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:01.008211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1122/2248] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:01.032281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecac595-0018-4497-afc9-84412d9139d6 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:01.040652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cecac595-0018-4497-afc9-84412d9139d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:01.108173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46383422ee4d44fdab36c49f904b8440', 'request_id': 'req-d607802d-200d-4625-a298-565c8d0597db'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:01.128409 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46383422ee4d44fdab36c49f904b8440', 'request_id': 'req-d607802d-200d-4625-a298-565c8d0597db'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:01.184236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecac595-0018-4497-afc9-84412d9139d6 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:01.233298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46383422ee4d44fdab36c49f904b8440', 'request_id': 'req-d607802d-200d-4625-a298-565c8d0597db'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:01.252201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] The token cache is being invalidate because project 46383422ee4d44fdab36c49f904b8440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:32:01.252722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:01.253972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d607802d-200d-4625-a298-565c8d0597db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:01.257352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1127/2249] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:32:00 2022] DELETE /identity/v3/projects/46383422ee4d44fdab36c49f904b8440 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:01.262756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30f663-fad5-4487-b6b6-9faff9971776 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:01.263727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30f663-fad5-4487-b6b6-9faff9971776 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:01.264166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30f663-fad5-4487-b6b6-9faff9971776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:01.296883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9b30f663-fad5-4487-b6b6-9faff9971776 None None] MFA Rules not processed for user `c8982a7c52314c5a82d80a03d9a833d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:01.299616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9b30f663-fad5-4487-b6b6-9faff9971776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:01.307884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cf473-bf43-49c4-8b4a-f4271e91e962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:01.308568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cf473-bf43-49c4-8b4a-f4271e91e962 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:01.308848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cf473-bf43-49c4-8b4a-f4271e91e962 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:01.309121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cf473-bf43-49c4-8b4a-f4271e91e962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:01.311777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-431cf473-bf43-49c4-8b4a-f4271e91e962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:01.412746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1128/2250] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:01.414609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee7359-23ab-4b07-915b-27ddf8d4eade None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:01.415071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee7359-23ab-4b07-915b-27ddf8d4eade None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:01.415409 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee7359-23ab-4b07-915b-27ddf8d4eade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:01.444760 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-36ee7359-23ab-4b07-915b-27ddf8d4eade None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:01.446158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-36ee7359-23ab-4b07-915b-27ddf8d4eade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:01.453853 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cf473-bf43-49c4-8b4a-f4271e91e962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:01.455645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431cf473-bf43-49c4-8b4a-f4271e91e962 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:01.491757 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1123/2251] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:32:01.492977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cf473-bf43-49c4-8b4a-f4271e91e962 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:01.495199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-431cf473-bf43-49c4-8b4a-f4271e91e962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:01.538954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:01.626582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1129/2252] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:01.636217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb0cd-505f-4c5a-8cff-ff90966dd739 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:01.636538 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb0cd-505f-4c5a-8cff-ff90966dd739 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:01.636943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb0cd-505f-4c5a-8cff-ff90966dd739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:01.640315 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cf473-bf43-49c4-8b4a-f4271e91e962 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:01.650223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:01.651791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:01.652559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:01.653364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] PATH_INFO: `/v3/users/67c97168b0484f70aeaf558b84b9f99c` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:01.656817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c97168b0484f70aeaf558b84b9f99c)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:01.660186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-abbeb0cd-505f-4c5a-8cff-ff90966dd739 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:01.661472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:01.661953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-abbeb0cd-505f-4c5a-8cff-ff90966dd739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:01.674784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1130/2253] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:01.676212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeb0cd-505f-4c5a-8cff-ff90966dd739 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:01.689274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeb0cd-505f-4c5a-8cff-ff90966dd739 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:01.698285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75eb106-c19b-4eaa-a61c-1c8ad226ff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:01.699052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75eb106-c19b-4eaa-a61c-1c8ad226ff86 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:01.699339 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75eb106-c19b-4eaa-a61c-1c8ad226ff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:01.699789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75eb106-c19b-4eaa-a61c-1c8ad226ff86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:01.700607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75eb106-c19b-4eaa-a61c-1c8ad226ff86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:32:01.701625 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75eb106-c19b-4eaa-a61c-1c8ad226ff86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:01.705396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75eb106-c19b-4eaa-a61c-1c8ad226ff86 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:01.724993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1131/2254] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:32:01.732518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75eb106-c19b-4eaa-a61c-1c8ad226ff86 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:01.762709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75eb106-c19b-4eaa-a61c-1c8ad226ff86 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:01.762709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6858138-bd3d-49b0-a111-e7a708232803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:01.762709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6858138-bd3d-49b0-a111-e7a708232803 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:01.762709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6858138-bd3d-49b0-a111-e7a708232803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:01.764624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6858138-bd3d-49b0-a111-e7a708232803 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:01.764624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6858138-bd3d-49b0-a111-e7a708232803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:01.764624 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6858138-bd3d-49b0-a111-e7a708232803 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:02.191866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1132/2255] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:32:01 2022] POST /identity/v3/projects => generated 369 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:32:02.193576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6858138-bd3d-49b0-a111-e7a708232803 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:02.197133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a6858138-bd3d-49b0-a111-e7a708232803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:02.228716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c97168b0484f70aeaf558b84b9f99c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:02.296912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6858138-bd3d-49b0-a111-e7a708232803 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:02.310184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f123f-869d-4b2d-9682-93e3b508d743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:02.311395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f123f-869d-4b2d-9682-93e3b508d743 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:02.312162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f123f-869d-4b2d-9682-93e3b508d743 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:02.312670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f123f-869d-4b2d-9682-93e3b508d743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:02.318504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c03f123f-869d-4b2d-9682-93e3b508d743 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:02.346519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c97168b0484f70aeaf558b84b9f99c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:02.349499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c97168b0484f70aeaf558b84b9f99c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:02.405515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c97168b0484f70aeaf558b84b9f99c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:02.406521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c97168b0484f70aeaf558b84b9f99c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:02.430879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1124/2256] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:32:01 2022] DELETE /identity/v3/users/67c97168b0484f70aeaf558b84b9f99c => generated 0 bytes in 938 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:02.438623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:02.438623 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:02.482440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f123f-869d-4b2d-9682-93e3b508d743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:02.490059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f123f-869d-4b2d-9682-93e3b508d743 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:02.515883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1133/2257] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:02.529930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3815f10-8cf6-4a1b-860f-28f0fcc702f0 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:02.530223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3815f10-8cf6-4a1b-860f-28f0fcc702f0 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:02.533764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3815f10-8cf6-4a1b-860f-28f0fcc702f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:02.556027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f177fd7-929a-45aa-958a-aa2e438ca95a admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:02.577847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf124cb-13f7-421d-a89e-5590ce614f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:02.577847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf124cb-13f7-421d-a89e-5590ce614f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:02.577847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf124cb-13f7-421d-a89e-5590ce614f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:02.577847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf124cb-13f7-421d-a89e-5590ce614f56 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:02.578467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf124cb-13f7-421d-a89e-5590ce614f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:02.583819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf124cb-13f7-421d-a89e-5590ce614f56 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:02.597685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f3815f10-8cf6-4a1b-860f-28f0fcc702f0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:02.600059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f3815f10-8cf6-4a1b-860f-28f0fcc702f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:02.628174 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1134/2258] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:02.636075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3815f10-8cf6-4a1b-860f-28f0fcc702f0 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:02.637872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f3815f10-8cf6-4a1b-860f-28f0fcc702f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:02.727133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3815f10-8cf6-4a1b-860f-28f0fcc702f0 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:02.732092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31993501-491b-4d49-93f5-6c651444fef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:02.748055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31993501-491b-4d49-93f5-6c651444fef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:02.748055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31993501-491b-4d49-93f5-6c651444fef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:02.748055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31993501-491b-4d49-93f5-6c651444fef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:02.748587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1125/2259] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:32:02 2022] POST /identity/v3/users => generated 285 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:32:02.751541 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf124cb-13f7-421d-a89e-5590ce614f56 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:02.772715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf124cb-13f7-421d-a89e-5590ce614f56 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:02.782880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31993501-491b-4d49-93f5-6c651444fef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:02.783807 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3774db-f1b3-4216-a8c9-45b321b7f0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:02.785274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3774db-f1b3-4216-a8c9-45b321b7f0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:02.785832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3774db-f1b3-4216-a8c9-45b321b7f0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:02.786320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3774db-f1b3-4216-a8c9-45b321b7f0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:02.787317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31993501-491b-4d49-93f5-6c651444fef2 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:02.787764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3774db-f1b3-4216-a8c9-45b321b7f0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:02.791787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3774db-f1b3-4216-a8c9-45b321b7f0c4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:02.814101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1135/2260] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:32:02.821272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1126/2261] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:32:02.833582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31993501-491b-4d49-93f5-6c651444fef2 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:02.855473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31993501-491b-4d49-93f5-6c651444fef2 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:02.861864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee19c1-0171-4edd-bfef-0339dad73d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:02.862661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee19c1-0171-4edd-bfef-0339dad73d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:02.863038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee19c1-0171-4edd-bfef-0339dad73d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:02.863402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee19c1-0171-4edd-bfef-0339dad73d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:02.864280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee19c1-0171-4edd-bfef-0339dad73d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:02.866042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee19c1-0171-4edd-bfef-0339dad73d39 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:02.907677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1136/2262] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:32:02.916507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3774db-f1b3-4216-a8c9-45b321b7f0c4 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:02.962128 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3774db-f1b3-4216-a8c9-45b321b7f0c4 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:02.986068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ca669-134e-4965-a894-1ca2021d7fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:02.989316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ca669-134e-4965-a894-1ca2021d7fd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:02.989891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ca669-134e-4965-a894-1ca2021d7fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:02.992122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ca669-134e-4965-a894-1ca2021d7fd9 admin admin] PATH_INFO: `/v3/projects/5a1bb8bc9a6542469dc619216bf570e8/users/ce82595e190e40268a8788a2de45e56c/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:03.041338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ca669-134e-4965-a894-1ca2021d7fd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1bb8bc9a6542469dc619216bf570e8, user_id=ce82595e190e40268a8788a2de45e56c, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:03.049177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ca669-134e-4965-a894-1ca2021d7fd9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:03.165898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1127/2263] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:02 2022] PUT /identity/v3/projects/5a1bb8bc9a6542469dc619216bf570e8/users/ce82595e190e40268a8788a2de45e56c/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:03.186451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea39895-26e4-4958-a04d-c1c46402973b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:03.186793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea39895-26e4-4958-a04d-c1c46402973b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:03.187083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea39895-26e4-4958-a04d-c1c46402973b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:03.244313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-dea39895-26e4-4958-a04d-c1c46402973b None None] MFA Rules not processed for user `ce82595e190e40268a8788a2de45e56c`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:03.245672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-dea39895-26e4-4958-a04d-c1c46402973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:03.295843 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:03.373912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1137/2264] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:32:03.385546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ca669-134e-4965-a894-1ca2021d7fd9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:03.387636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-541ca669-134e-4965-a894-1ca2021d7fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:03.402911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea39895-26e4-4958-a04d-c1c46402973b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:03.418660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-dea39895-26e4-4958-a04d-c1c46402973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:03.574478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea39895-26e4-4958-a04d-c1c46402973b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:03.583803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:03.585749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:03.586537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:03.586990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] PATH_INFO: `/v3/users/eec4e0e64cd3466db7295a98921df777` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:03.588501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec4e0e64cd3466db7295a98921df777)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:03.598084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:03.613568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ca669-134e-4965-a894-1ca2021d7fd9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:03.645335 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6fc44-35ee-482b-82eb-edc31a82412a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f10c258fe5f45ccab1669fa32ed6029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:03.646533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6fc44-35ee-482b-82eb-edc31a82412a service glance] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:03.646824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6fc44-35ee-482b-82eb-edc31a82412a service glance] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:03.647126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6fc44-35ee-482b-82eb-edc31a82412a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:03.658582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6fc44-35ee-482b-82eb-edc31a82412a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:03.661153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6fc44-35ee-482b-82eb-edc31a82412a service glance] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:03.672871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1128/2265] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:03.823309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6fc44-35ee-482b-82eb-edc31a82412a service glance] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:03.829170 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c9f6fc44-35ee-482b-82eb-edc31a82412a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:03.854931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6fc44-35ee-482b-82eb-edc31a82412a service glance] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:03.866800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:03.867706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:03.868292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:03.868630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] PATH_INFO: `/v3/projects/9903ae33db5a4002afc782eb4fba7612` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:03.870665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9903ae33db5a4002afc782eb4fba7612)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:03.880334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:03.943780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec4e0e64cd3466db7295a98921df777'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:03.946008 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec4e0e64cd3466db7295a98921df777'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:03.947210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec4e0e64cd3466db7295a98921df777'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:03.966021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec4e0e64cd3466db7295a98921df777'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.029729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d88ac756-8ce6-4b42-853c-1657185f034a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec4e0e64cd3466db7295a98921df777'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.065064 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9903ae33db5a4002afc782eb4fba7612', 'request_id': 'req-3e7691b3-addf-474f-98af-37fa1a1e1d3f'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.083681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9903ae33db5a4002afc782eb4fba7612', 'request_id': 'req-3e7691b3-addf-474f-98af-37fa1a1e1d3f'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.181312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1138/2266] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:32:03 2022] DELETE /identity/v3/users/eec4e0e64cd3466db7295a98921df777 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:04.192435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ca8cd-cc14-4100-b13b-2a43a239e9f0 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:04.193075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ca8cd-cc14-4100-b13b-2a43a239e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:04.193583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ca8cd-cc14-4100-b13b-2a43a239e9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:04.247365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-2d4ca8cd-cc14-4100-b13b-2a43a239e9f0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:04.247365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2d4ca8cd-cc14-4100-b13b-2a43a239e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:04.269484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9903ae33db5a4002afc782eb4fba7612', 'request_id': 'req-3e7691b3-addf-474f-98af-37fa1a1e1d3f'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.311909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:04.327469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] The token cache is being invalidate because project 9903ae33db5a4002afc782eb4fba7612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:32:04.328486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.333085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.333085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1129/2267] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:32:03 2022] DELETE /identity/v3/projects/9903ae33db5a4002afc782eb4fba7612 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:04.337117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:04.339295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:04.398180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1139/2268] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:04.411253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ca8cd-cc14-4100-b13b-2a43a239e9f0 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:04.412918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2d4ca8cd-cc14-4100-b13b-2a43a239e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:04.471016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7691b3-addf-474f-98af-37fa1a1e1d3f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:04.478115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:04.479265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:04.481166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:04.481166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] PATH_INFO: `/v3/users/6c4879d538d14286a0eddbb9f4707593` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:04.483866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4879d538d14286a0eddbb9f4707593)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:04.486995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:04.524938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ca8cd-cc14-4100-b13b-2a43a239e9f0 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:04.530157 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1278f-aac5-4829-a78d-d495e920199d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:04.530850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1278f-aac5-4829-a78d-d495e920199d service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:04.531133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1278f-aac5-4829-a78d-d495e920199d service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:04.531497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1278f-aac5-4829-a78d-d495e920199d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:04.543300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1278f-aac5-4829-a78d-d495e920199d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:04.545749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1278f-aac5-4829-a78d-d495e920199d service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:04.576299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1140/2269] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:04.781007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4879d538d14286a0eddbb9f4707593'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.917967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4879d538d14286a0eddbb9f4707593'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.918722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4879d538d14286a0eddbb9f4707593'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.945171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4879d538d14286a0eddbb9f4707593'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.946714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4879d538d14286a0eddbb9f4707593'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:04.961119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1130/2270] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:32:04 2022] DELETE /identity/v3/users/6c4879d538d14286a0eddbb9f4707593 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:05.301547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1278f-aac5-4829-a78d-d495e920199d service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:05.304518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4ad1278f-aac5-4829-a78d-d495e920199d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:05.330549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1278f-aac5-4829-a78d-d495e920199d service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:05.490117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:05.491491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:05.492212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:05.493982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] PATH_INFO: `/v3/projects/de0dcb7a70894391a6c06acb0a2e0eae` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:05.496653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] RBAC: Authorizing `identity:delete_project(project_id=de0dcb7a70894391a6c06acb0a2e0eae)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:05.500505 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:05.742166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:05.755121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:05.755997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de0dcb7a70894391a6c06acb0a2e0eae', 'request_id': 'req-b5077e46-58c0-4843-bead-0e67967cd18c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:05.760927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:05.773054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de0dcb7a70894391a6c06acb0a2e0eae', 'request_id': 'req-b5077e46-58c0-4843-bead-0e67967cd18c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:05.787577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1ac58-a92e-4a84-ad2d-b54f0e372bc6 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:05.870548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de0dcb7a70894391a6c06acb0a2e0eae', 'request_id': 'req-b5077e46-58c0-4843-bead-0e67967cd18c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:05.886666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] The token cache is being invalidate because project de0dcb7a70894391a6c06acb0a2e0eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:32:05.886972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:05.887742 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:05.890416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1141/2271] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:32:05 2022] DELETE /identity/v3/projects/de0dcb7a70894391a6c06acb0a2e0eae => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:05.895009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:05.897127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:05.898325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:05.910774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:05.911421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:05.911796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:05.912071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] PATH_INFO: `/v3/projects/26ed1108cdc04b768050eb65b0159659` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:05.913542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26ed1108cdc04b768050eb65b0159659)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:05.916311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:06.006265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5077e46-58c0-4843-bead-0e67967cd18c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:06.027143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:06.027143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:06.027143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:06.027143 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] PATH_INFO: `/v3/users/f43b25af4f724e819af599b3bb332e8e` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:06.027517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f43b25af4f724e819af599b3bb332e8e)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:06.027517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:06.116429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26ed1108cdc04b768050eb65b0159659', 'request_id': 'req-6ed8e55b-920a-4dc1-97d9-efd544078bf9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:06.123759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26ed1108cdc04b768050eb65b0159659', 'request_id': 'req-6ed8e55b-920a-4dc1-97d9-efd544078bf9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:06.192781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26ed1108cdc04b768050eb65b0159659', 'request_id': 'req-6ed8e55b-920a-4dc1-97d9-efd544078bf9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:06.208520 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] The token cache is being invalidate because project 26ed1108cdc04b768050eb65b0159659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:32:06.208918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:06.209639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6ed8e55b-920a-4dc1-97d9-efd544078bf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:06.211405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1131/2272] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:32:05 2022] DELETE /identity/v3/projects/26ed1108cdc04b768050eb65b0159659 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:06.217402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83948a6-e7d4-4d22-8e61-9e7e11b2021c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:06.217674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83948a6-e7d4-4d22-8e61-9e7e11b2021c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:06.218125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83948a6-e7d4-4d22-8e61-9e7e11b2021c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:06.243193 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a83948a6-e7d4-4d22-8e61-9e7e11b2021c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:06.244513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a83948a6-e7d4-4d22-8e61-9e7e11b2021c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:06.351796 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f43b25af4f724e819af599b3bb332e8e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:06.353065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f43b25af4f724e819af599b3bb332e8e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:06.354322 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f43b25af4f724e819af599b3bb332e8e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:06.358271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1132/2273] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:06.366293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f43b25af4f724e819af599b3bb332e8e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:06.370321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6472-5b50-40fb-a9ed-5be543832b3d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:06.370927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6472-5b50-40fb-a9ed-5be543832b3d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:06.371390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb6472-5b50-40fb-a9ed-5be543832b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:06.397570 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f43b25af4f724e819af599b3bb332e8e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:06.399358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-29bb6472-5b50-40fb-a9ed-5be543832b3d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:06.400966 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-29bb6472-5b50-40fb-a9ed-5be543832b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:06.414525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1133/2274] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:06.417643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb6472-5b50-40fb-a9ed-5be543832b3d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:06.432011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb6472-5b50-40fb-a9ed-5be543832b3d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:06.439308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8a024-019e-4f06-a280-31e5591baef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:06.440204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a024-019e-4f06-a280-31e5591baef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:06.440916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a024-019e-4f06-a280-31e5591baef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:06.441556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a024-019e-4f06-a280-31e5591baef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:06.442657 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8a024-019e-4f06-a280-31e5591baef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:32:06.443145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8a024-019e-4f06-a280-31e5591baef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:06.444944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8a024-019e-4f06-a280-31e5591baef4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:06.468000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1134/2275] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:32:06.482057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8a024-019e-4f06-a280-31e5591baef4 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:06.502988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8a024-019e-4f06-a280-31e5591baef4 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:06.509990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230343b-b568-43fe-8b01-921820d4986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:06.510461 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230343b-b568-43fe-8b01-921820d4986f admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:06.510941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230343b-b568-43fe-8b01-921820d4986f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:06.511277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2230343b-b568-43fe-8b01-921820d4986f admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:06.512047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230343b-b568-43fe-8b01-921820d4986f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:06.514719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2230343b-b568-43fe-8b01-921820d4986f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:06.571457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1142/2276] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:32:05 2022] DELETE /identity/v3/users/f43b25af4f724e819af599b3bb332e8e => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:06.728939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1135/2277] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:32:06 2022] POST /identity/v3/projects => generated 373 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:32:06.733546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:06.839464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91682fbb-9030-49a3-9e98-6030b7561ee2 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:06.975424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd897e-86b4-4929-8727-1cfa0906d0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:06.976411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd897e-86b4-4929-8727-1cfa0906d0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:06.983754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd897e-86b4-4929-8727-1cfa0906d0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:06.983754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd897e-86b4-4929-8727-1cfa0906d0f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:06.983754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd897e-86b4-4929-8727-1cfa0906d0f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:06.983754 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd897e-86b4-4929-8727-1cfa0906d0f1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:07.117778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1143/2278] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:32:06 2022] POST /identity/v3/users => generated 287 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:32:07.123145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230343b-b568-43fe-8b01-921820d4986f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:07.144508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2230343b-b568-43fe-8b01-921820d4986f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:07.150000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbaf4d3-1919-4d54-a81e-08bd34a5d62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:07.150901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaf4d3-1919-4d54-a81e-08bd34a5d62e admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:07.151188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaf4d3-1919-4d54-a81e-08bd34a5d62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:07.151462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaf4d3-1919-4d54-a81e-08bd34a5d62e admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:07.152188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbaf4d3-1919-4d54-a81e-08bd34a5d62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:07.154090 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbaf4d3-1919-4d54-a81e-08bd34a5d62e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:07.175320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1136/2279] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:32:07.182122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd897e-86b4-4929-8727-1cfa0906d0f1 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:07.213033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd897e-86b4-4929-8727-1cfa0906d0f1 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:07.224755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee2a2c-7745-48c0-9d54-55b48f3a5761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:07.227152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee2a2c-7745-48c0-9d54-55b48f3a5761 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:07.234617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee2a2c-7745-48c0-9d54-55b48f3a5761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:07.234617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee2a2c-7745-48c0-9d54-55b48f3a5761 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:07.234617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee2a2c-7745-48c0-9d54-55b48f3a5761 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:07.234617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee2a2c-7745-48c0-9d54-55b48f3a5761 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:07.265482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1144/2280] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:32:07.265849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbaf4d3-1919-4d54-a81e-08bd34a5d62e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:07.281663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbaf4d3-1919-4d54-a81e-08bd34a5d62e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:07.289517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce97b3a-d9fc-4242-a7c0-e983ec4bcfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:07.299783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce97b3a-d9fc-4242-a7c0-e983ec4bcfd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:07.299783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce97b3a-d9fc-4242-a7c0-e983ec4bcfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:07.299783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce97b3a-d9fc-4242-a7c0-e983ec4bcfd2 admin admin] PATH_INFO: `/v3/projects/4240f3dbafbb4d8abbf79d6e8a2648dd/users/96f28349adf24ae7acfa9d02fddfcf98/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:07.312079 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce97b3a-d9fc-4242-a7c0-e983ec4bcfd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4240f3dbafbb4d8abbf79d6e8a2648dd, user_id=96f28349adf24ae7acfa9d02fddfcf98, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:07.316922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce97b3a-d9fc-4242-a7c0-e983ec4bcfd2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:07.386743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee2a2c-7745-48c0-9d54-55b48f3a5761 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:07.389708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-84ee2a2c-7745-48c0-9d54-55b48f3a5761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:07.442106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1137/2281] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:07 2022] PUT /identity/v3/projects/4240f3dbafbb4d8abbf79d6e8a2648dd/users/96f28349adf24ae7acfa9d02fddfcf98/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:07.454629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666924-f000-4ccf-9183-d2690debbfdd None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:07.455031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666924-f000-4ccf-9183-d2690debbfdd None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:07.455613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2666924-f000-4ccf-9183-d2690debbfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:07.458857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee2a2c-7745-48c0-9d54-55b48f3a5761 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:07.485116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-c2666924-f000-4ccf-9183-d2690debbfdd None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:07.486789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c2666924-f000-4ccf-9183-d2690debbfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:07.561137 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:07.562005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:07.562292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:07.562749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] PATH_INFO: `/v3/projects/a5b5b831866440918b5edaf2a09bc8be` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:07.564090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b5b831866440918b5edaf2a09bc8be)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:07.566261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:07.580544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1138/2282] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:07.593726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2666924-f000-4ccf-9183-d2690debbfdd None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:07.595159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c2666924-f000-4ccf-9183-d2690debbfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:07.677131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2666924-f000-4ccf-9183-d2690debbfdd None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:07.683305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348867c6-0dfa-4d8c-816f-c7ef0d125599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:07.684286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348867c6-0dfa-4d8c-816f-c7ef0d125599 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:07.684742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348867c6-0dfa-4d8c-816f-c7ef0d125599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:07.685182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348867c6-0dfa-4d8c-816f-c7ef0d125599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:07.697430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348867c6-0dfa-4d8c-816f-c7ef0d125599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:07.699196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348867c6-0dfa-4d8c-816f-c7ef0d125599 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:07.711174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1139/2283] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:07.754655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b5b831866440918b5edaf2a09bc8be', 'request_id': 'req-77423b22-9ba6-4b64-93b2-34224bfe2e01'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:07.766383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b5b831866440918b5edaf2a09bc8be', 'request_id': 'req-77423b22-9ba6-4b64-93b2-34224bfe2e01'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:07.884703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b5b831866440918b5edaf2a09bc8be', 'request_id': 'req-77423b22-9ba6-4b64-93b2-34224bfe2e01'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:07.905587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] The token cache is being invalidate because project a5b5b831866440918b5edaf2a09bc8be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:32:07.905896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:07.906849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-77423b22-9ba6-4b64-93b2-34224bfe2e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:07.908656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1145/2284] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:32:07 2022] DELETE /identity/v3/projects/a5b5b831866440918b5edaf2a09bc8be => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:07.914885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c55a5-920b-4012-9b7f-bd6462462410 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:07.915166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c55a5-920b-4012-9b7f-bd6462462410 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:07.915433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c55a5-920b-4012-9b7f-bd6462462410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:07.945474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-499c55a5-920b-4012-9b7f-bd6462462410 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:07.947728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-499c55a5-920b-4012-9b7f-bd6462462410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:08.052842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1140/2285] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:32:08.066559 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5e4fb-4d56-4559-a93e-fce178183898 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:08.066856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5e4fb-4d56-4559-a93e-fce178183898 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:08.067117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5e4fb-4d56-4559-a93e-fce178183898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:08.089800 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-8eb5e4fb-4d56-4559-a93e-fce178183898 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:08.090989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-8eb5e4fb-4d56-4559-a93e-fce178183898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:08.099831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1146/2286] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:08.105318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c55a5-920b-4012-9b7f-bd6462462410 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:08.124413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c55a5-920b-4012-9b7f-bd6462462410 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:08.132560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074c1f4-9306-471f-ae40-21a756451560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:08.133714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074c1f4-9306-471f-ae40-21a756451560 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:08.134551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074c1f4-9306-471f-ae40-21a756451560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:08.135081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074c1f4-9306-471f-ae40-21a756451560 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:08.136772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074c1f4-9306-471f-ae40-21a756451560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:32:08.138026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074c1f4-9306-471f-ae40-21a756451560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:08.142105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3074c1f4-9306-471f-ae40-21a756451560 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:08.168949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1141/2287] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:32:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:32:08.181442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5e4fb-4d56-4559-a93e-fce178183898 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:08.206655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5e4fb-4d56-4559-a93e-fce178183898 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:08.214979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807de39e-5f6c-4313-8673-e00338cce110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:08.216013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807de39e-5f6c-4313-8673-e00338cce110 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:08.216502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807de39e-5f6c-4313-8673-e00338cce110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:08.217201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807de39e-5f6c-4313-8673-e00338cce110 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:08.218794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807de39e-5f6c-4313-8673-e00338cce110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:08.222258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807de39e-5f6c-4313-8673-e00338cce110 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:08.361006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1147/2288] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:32:08 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:32:08.365830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074c1f4-9306-471f-ae40-21a756451560 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:08.378652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074c1f4-9306-471f-ae40-21a756451560 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:08.488221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e42b08-e553-48dd-b4e2-ebc59e71e500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:08.489512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e42b08-e553-48dd-b4e2-ebc59e71e500 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:08.490078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e42b08-e553-48dd-b4e2-ebc59e71e500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:08.490619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e42b08-e553-48dd-b4e2-ebc59e71e500 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:08.491477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e42b08-e553-48dd-b4e2-ebc59e71e500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:08.494231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e42b08-e553-48dd-b4e2-ebc59e71e500 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:08.651989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1142/2289] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:32:08 2022] POST /identity/v3/users => generated 288 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:32:08.657939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807de39e-5f6c-4313-8673-e00338cce110 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:08.674706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807de39e-5f6c-4313-8673-e00338cce110 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:08.687693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82541951-7fd8-4602-9277-516f401404e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:08.691316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82541951-7fd8-4602-9277-516f401404e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:08.691316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82541951-7fd8-4602-9277-516f401404e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:08.691316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82541951-7fd8-4602-9277-516f401404e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:08.691316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82541951-7fd8-4602-9277-516f401404e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:08.693473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82541951-7fd8-4602-9277-516f401404e0 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:08.706711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:08.716005 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1148/2290] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:32:08.721655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e42b08-e553-48dd-b4e2-ebc59e71e500 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:08.733371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e42b08-e553-48dd-b4e2-ebc59e71e500 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:08.739344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f02e4-9799-46d8-b346-e3f0b861a18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:08.740114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f02e4-9799-46d8-b346-e3f0b861a18a admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:08.740424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f02e4-9799-46d8-b346-e3f0b861a18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:08.740847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f02e4-9799-46d8-b346-e3f0b861a18a admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:08.741519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f02e4-9799-46d8-b346-e3f0b861a18a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:08.743161 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f02e4-9799-46d8-b346-e3f0b861a18a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:08.757425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1143/2291] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:32:08.762310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82541951-7fd8-4602-9277-516f401404e0 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:08.774737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82541951-7fd8-4602-9277-516f401404e0 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:08.787310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fc5e2-8544-46ed-9d3f-c83a13aa98d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:08.792768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fc5e2-8544-46ed-9d3f-c83a13aa98d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:08.793293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fc5e2-8544-46ed-9d3f-c83a13aa98d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:08.793934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fc5e2-8544-46ed-9d3f-c83a13aa98d8 admin admin] PATH_INFO: `/v3/projects/8cf171d42bd64851869a81c2a0c82ddf/users/c6075349ebdc4caf88f8ae966a8d76f6/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:08.808677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:08.826449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fc5e2-8544-46ed-9d3f-c83a13aa98d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf171d42bd64851869a81c2a0c82ddf, user_id=c6075349ebdc4caf88f8ae966a8d76f6, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:08.829418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fc5e2-8544-46ed-9d3f-c83a13aa98d8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:08.998247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1149/2292] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:08 2022] PUT /identity/v3/projects/8cf171d42bd64851869a81c2a0c82ddf/users/c6075349ebdc4caf88f8ae966a8d76f6/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:09.001630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5958bb0-28a4-4629-8687-92b5009da530 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:09.002032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5958bb0-28a4-4629-8687-92b5009da530 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:09.002391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5958bb0-28a4-4629-8687-92b5009da530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:09.051034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f5958bb0-28a4-4629-8687-92b5009da530 None None] MFA Rules not processed for user `c6075349ebdc4caf88f8ae966a8d76f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:09.052554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f5958bb0-28a4-4629-8687-92b5009da530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:09.179971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1144/2293] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:09.181209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fc5e2-8544-46ed-9d3f-c83a13aa98d8 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:09.287205 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fc5e2-8544-46ed-9d3f-c83a13aa98d8 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:09.293500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74387e-9739-413e-805f-251d240048f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:09.294565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74387e-9739-413e-805f-251d240048f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:09.294855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74387e-9739-413e-805f-251d240048f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:09.295142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74387e-9739-413e-805f-251d240048f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:09.296095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74387e-9739-413e-805f-251d240048f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:09.298114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74387e-9739-413e-805f-251d240048f1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:09.465164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1150/2294] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:32:09 2022] POST /identity/v3/projects => generated 375 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:32:09.468863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5958bb0-28a4-4629-8687-92b5009da530 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:09.483251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5958bb0-28a4-4629-8687-92b5009da530 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:09.598608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25d566-9c78-4b90-be7f-f77b59f587af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:09.599260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d566-9c78-4b90-be7f-f77b59f587af admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:09.599543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d566-9c78-4b90-be7f-f77b59f587af admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:09.599847 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25d566-9c78-4b90-be7f-f77b59f587af admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:09.600841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25d566-9c78-4b90-be7f-f77b59f587af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:09.602889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25d566-9c78-4b90-be7f-f77b59f587af admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:09.642659 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbb919-07dc-4d65-9d6c-a51733f6c62d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:09.643075 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbb919-07dc-4d65-9d6c-a51733f6c62d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:09.643466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbb919-07dc-4d65-9d6c-a51733f6c62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:09.693948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5cbbb919-07dc-4d65-9d6c-a51733f6c62d None None] MFA Rules not processed for user `96f28349adf24ae7acfa9d02fddfcf98`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:09.696601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5cbbb919-07dc-4d65-9d6c-a51733f6c62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:09.704434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1145/2295] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:32:09 2022] POST /identity/v3/users => generated 294 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:32:09.706241 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25d566-9c78-4b90-be7f-f77b59f587af admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:09.734342 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25d566-9c78-4b90-be7f-f77b59f587af admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:09.758178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e7979-5872-4da2-b2ed-24cea23781d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:09.758815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e7979-5872-4da2-b2ed-24cea23781d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:09.759097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e7979-5872-4da2-b2ed-24cea23781d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:09.759374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e7979-5872-4da2-b2ed-24cea23781d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:09.760822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e7979-5872-4da2-b2ed-24cea23781d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:09.762517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e7979-5872-4da2-b2ed-24cea23781d1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:09.810555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1146/2296] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:32:09.811872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e7979-5872-4da2-b2ed-24cea23781d1 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:09.842787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e7979-5872-4da2-b2ed-24cea23781d1 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:09.864476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94e7cb-bc5c-493e-b8fd-70eba81967a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:09.868553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94e7cb-bc5c-493e-b8fd-70eba81967a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:09.868553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94e7cb-bc5c-493e-b8fd-70eba81967a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:09.868553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b94e7cb-bc5c-493e-b8fd-70eba81967a7 admin admin] PATH_INFO: `/v3/projects/fabdc17fa59b4ab0a1b36bda87831cfb/users/c7472a113f494acfaac58182a21fc422/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:09.902528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1151/2297] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:32:09.905102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94e7cb-bc5c-493e-b8fd-70eba81967a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabdc17fa59b4ab0a1b36bda87831cfb, user_id=c7472a113f494acfaac58182a21fc422, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:09.908706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b94e7cb-bc5c-493e-b8fd-70eba81967a7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:09.909870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbb919-07dc-4d65-9d6c-a51733f6c62d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:09.912715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5cbbb919-07dc-4d65-9d6c-a51733f6c62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:09.948306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbb919-07dc-4d65-9d6c-a51733f6c62d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:09.959055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ecc47-c12b-4c52-9db6-f22a1f623449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:09.960220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ecc47-c12b-4c52-9db6-f22a1f623449 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:09.960757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ecc47-c12b-4c52-9db6-f22a1f623449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:09.961392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ecc47-c12b-4c52-9db6-f22a1f623449 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:09.962973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ecc47-c12b-4c52-9db6-f22a1f623449 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:09.975803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ecc47-c12b-4c52-9db6-f22a1f623449 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:10.055772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1147/2298] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:09 2022] PUT /identity/v3/projects/fabdc17fa59b4ab0a1b36bda87831cfb/users/c7472a113f494acfaac58182a21fc422/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:10.060670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94e7cb-bc5c-493e-b8fd-70eba81967a7 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:10.117325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1152/2299] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:32:09 2022] POST /identity/v3/projects => generated 371 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:32:10.121921 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ecc47-c12b-4c52-9db6-f22a1f623449 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:10.164722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b94e7cb-bc5c-493e-b8fd-70eba81967a7 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:10.240314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ecc47-c12b-4c52-9db6-f22a1f623449 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:10.249265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1901fae-0f44-4dc3-a979-e3c86e84632f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:10.250397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1901fae-0f44-4dc3-a979-e3c86e84632f admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:10.250907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1901fae-0f44-4dc3-a979-e3c86e84632f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:10.251955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1901fae-0f44-4dc3-a979-e3c86e84632f admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:10.253434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1901fae-0f44-4dc3-a979-e3c86e84632f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:10.257653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1901fae-0f44-4dc3-a979-e3c86e84632f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:10.279495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794e0e5-0282-4e60-8981-f39c4f525ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:10.280960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794e0e5-0282-4e60-8981-f39c4f525ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:10.281540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794e0e5-0282-4e60-8981-f39c4f525ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:10.281961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794e0e5-0282-4e60-8981-f39c4f525ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:10.283217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794e0e5-0282-4e60-8981-f39c4f525ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:10.286746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794e0e5-0282-4e60-8981-f39c4f525ffc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:10.305450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1148/2300] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:32:10.310194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794e0e5-0282-4e60-8981-f39c4f525ffc admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:10.321790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794e0e5-0282-4e60-8981-f39c4f525ffc admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:10.328786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea178bf5-8446-43df-8787-73f5cf3bb9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:10.329780 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea178bf5-8446-43df-8787-73f5cf3bb9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:10.335044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea178bf5-8446-43df-8787-73f5cf3bb9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:10.335044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea178bf5-8446-43df-8787-73f5cf3bb9cd admin admin] PATH_INFO: `/v3/projects/fabdc17fa59b4ab0a1b36bda87831cfb/users/c7472a113f494acfaac58182a21fc422/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:10.335044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea178bf5-8446-43df-8787-73f5cf3bb9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabdc17fa59b4ab0a1b36bda87831cfb, user_id=c7472a113f494acfaac58182a21fc422, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:10.335897 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea178bf5-8446-43df-8787-73f5cf3bb9cd admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:10.359812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1149/2301] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:10 2022] PUT /identity/v3/projects/fabdc17fa59b4ab0a1b36bda87831cfb/users/c7472a113f494acfaac58182a21fc422/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:32:10.369444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1153/2302] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:32:10 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:32:10.371290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1901fae-0f44-4dc3-a979-e3c86e84632f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:10.372163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf2383-2481-4369-a052-8dcb01259cbc None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:10.372828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf2383-2481-4369-a052-8dcb01259cbc None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:10.373341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf2383-2481-4369-a052-8dcb01259cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:10.390704 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1901fae-0f44-4dc3-a979-e3c86e84632f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:10.408086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:10.427427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-34bf2383-2481-4369-a052-8dcb01259cbc None None] MFA Rules not processed for user `c7472a113f494acfaac58182a21fc422`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:10.429691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-34bf2383-2481-4369-a052-8dcb01259cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:10.474391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae2b39-b885-4d0e-96a8-05bbe7489b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:10.475324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae2b39-b885-4d0e-96a8-05bbe7489b33 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:10.475925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae2b39-b885-4d0e-96a8-05bbe7489b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:10.476326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae2b39-b885-4d0e-96a8-05bbe7489b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:10.477263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae2b39-b885-4d0e-96a8-05bbe7489b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:10.479919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae2b39-b885-4d0e-96a8-05bbe7489b33 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:10.499028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1154/2303] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:32:10.505027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae2b39-b885-4d0e-96a8-05bbe7489b33 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:10.520334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae2b39-b885-4d0e-96a8-05bbe7489b33 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:10.525998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee828628-256d-4997-b9f8-8977431dc1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:10.526693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee828628-256d-4997-b9f8-8977431dc1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:10.526972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee828628-256d-4997-b9f8-8977431dc1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:10.527259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee828628-256d-4997-b9f8-8977431dc1cf admin admin] PATH_INFO: `/v3/projects/97978ed6ec814ea89cbf6721409cfd00/users/d0a24e20d5ec48cfb5936380e0914a7d/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:10.536340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1150/2304] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:10.549764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf2383-2481-4369-a052-8dcb01259cbc None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:10.551053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee828628-256d-4997-b9f8-8977431dc1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=97978ed6ec814ea89cbf6721409cfd00, user_id=d0a24e20d5ec48cfb5936380e0914a7d, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:10.552099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-34bf2383-2481-4369-a052-8dcb01259cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:10.556697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee828628-256d-4997-b9f8-8977431dc1cf admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:10.643786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf2383-2481-4369-a052-8dcb01259cbc None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:10.650686 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8fc0b-a833-4b6c-befa-7cac4a167dba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:10.651902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8fc0b-a833-4b6c-befa-7cac4a167dba service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:10.652287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8fc0b-a833-4b6c-befa-7cac4a167dba service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:10.652792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8fc0b-a833-4b6c-befa-7cac4a167dba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:10.720078 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1155/2305] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:10 2022] PUT /identity/v3/projects/97978ed6ec814ea89cbf6721409cfd00/users/d0a24e20d5ec48cfb5936380e0914a7d/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:32:10.722151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee828628-256d-4997-b9f8-8977431dc1cf admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:10.773958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8fc0b-a833-4b6c-befa-7cac4a167dba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:10.776477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8fc0b-a833-4b6c-befa-7cac4a167dba service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:10.809176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1151/2306] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:10.836288 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee828628-256d-4997-b9f8-8977431dc1cf admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:10.840453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37f3bff-bf39-4cc6-9932-0f4da81ba91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:10.841608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f3bff-bf39-4cc6-9932-0f4da81ba91c admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:10.842200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f3bff-bf39-4cc6-9932-0f4da81ba91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:10.842419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f3bff-bf39-4cc6-9932-0f4da81ba91c admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:10.843860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f3bff-bf39-4cc6-9932-0f4da81ba91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:10.844748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37f3bff-bf39-4cc6-9932-0f4da81ba91c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:10.856851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1156/2307] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:32:10.861785 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8fc0b-a833-4b6c-befa-7cac4a167dba service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:10.878351 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8fc0b-a833-4b6c-befa-7cac4a167dba service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:10.886919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886e334-5eaa-4469-b60d-5bda84de224a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:10.888663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886e334-5eaa-4469-b60d-5bda84de224a admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:10.889200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886e334-5eaa-4469-b60d-5bda84de224a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:10.890514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886e334-5eaa-4469-b60d-5bda84de224a admin admin] PATH_INFO: `/v3/projects/97978ed6ec814ea89cbf6721409cfd00/users/d0a24e20d5ec48cfb5936380e0914a7d/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:10.893280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886e334-5eaa-4469-b60d-5bda84de224a admin admin] RBAC: Authorizing `identity:create_grant(project_id=97978ed6ec814ea89cbf6721409cfd00, user_id=d0a24e20d5ec48cfb5936380e0914a7d, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:10.898191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886e334-5eaa-4469-b60d-5bda84de224a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:10.930346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1152/2308] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:10 2022] PUT /identity/v3/projects/97978ed6ec814ea89cbf6721409cfd00/users/d0a24e20d5ec48cfb5936380e0914a7d/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:32:11.656148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:12.235788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:12.953425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43720c2-ebc4-445d-bd72-3cf37b3547a5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:12.962764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43720c2-ebc4-445d-bd72-3cf37b3547a5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:12.962764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43720c2-ebc4-445d-bd72-3cf37b3547a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:13.085323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b43720c2-ebc4-445d-bd72-3cf37b3547a5 None None] MFA Rules not processed for user `d0a24e20d5ec48cfb5936380e0914a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:13.087086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b43720c2-ebc4-445d-bd72-3cf37b3547a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:13.394783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1157/2309] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:13.518135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886e334-5eaa-4469-b60d-5bda84de224a admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:13.520882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a886e334-5eaa-4469-b60d-5bda84de224a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:13.614239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886e334-5eaa-4469-b60d-5bda84de224a admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:13.619692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b1c52-c0a0-4c1b-8763-6268b4fdd47a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:13.620740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b1c52-c0a0-4c1b-8763-6268b4fdd47a service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:13.621242 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b1c52-c0a0-4c1b-8763-6268b4fdd47a service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:13.621996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b1c52-c0a0-4c1b-8763-6268b4fdd47a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:13.715527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b1c52-c0a0-4c1b-8763-6268b4fdd47a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:13.718303 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b1c52-c0a0-4c1b-8763-6268b4fdd47a service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:13.744690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1153/2310] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:14.978782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43720c2-ebc4-445d-bd72-3cf37b3547a5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:15.186229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43720c2-ebc4-445d-bd72-3cf37b3547a5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:15.199498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:15.201261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:15.202214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:15.202935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] PATH_INFO: `/v3/users/c6075349ebdc4caf88f8ae966a8d76f6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:15.205918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6075349ebdc4caf88f8ae966a8d76f6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:15.213313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:15.563898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6075349ebdc4caf88f8ae966a8d76f6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:15.564658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6075349ebdc4caf88f8ae966a8d76f6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:15.565439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6075349ebdc4caf88f8ae966a8d76f6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:15.574762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6075349ebdc4caf88f8ae966a8d76f6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:15.589886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6075349ebdc4caf88f8ae966a8d76f6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:15.775315 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:15.783906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1158/2311] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:32:14 2022] DELETE /identity/v3/users/c6075349ebdc4caf88f8ae966a8d76f6 => generated 0 bytes in 807 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:15.788629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e036524-5f9b-4185-9e5b-bf8b2364065e None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:15.788999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e036524-5f9b-4185-9e5b-bf8b2364065e None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:15.789344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e036524-5f9b-4185-9e5b-bf8b2364065e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:15.809737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-3e036524-5f9b-4185-9e5b-bf8b2364065e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:15.811997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3e036524-5f9b-4185-9e5b-bf8b2364065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:15.907297 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1154/2312] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:15.927781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:15.930170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:16.071695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035734a6-9c41-4f1b-a764-27fd02a46998 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:16.071695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fa874-e4a2-4e57-bc58-1d4b49c3955f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:16.072554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fa874-e4a2-4e57-bc58-1d4b49c3955f service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:16.072952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fa874-e4a2-4e57-bc58-1d4b49c3955f service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:16.073327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fa874-e4a2-4e57-bc58-1d4b49c3955f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:16.094672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fa874-e4a2-4e57-bc58-1d4b49c3955f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:16.096517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fa874-e4a2-4e57-bc58-1d4b49c3955f service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:16.131461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1159/2313] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:16.236479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e036524-5f9b-4185-9e5b-bf8b2364065e None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:16.238169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3e036524-5f9b-4185-9e5b-bf8b2364065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:16.355109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e036524-5f9b-4185-9e5b-bf8b2364065e None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:16.361445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09b87d-4cab-4af8-a589-c1f0062d4f95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:16.362965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b87d-4cab-4af8-a589-c1f0062d4f95 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:16.363278 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b87d-4cab-4af8-a589-c1f0062d4f95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:16.363989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09b87d-4cab-4af8-a589-c1f0062d4f95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:16.367179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-da09b87d-4cab-4af8-a589-c1f0062d4f95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:16.450152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:16.508762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09b87d-4cab-4af8-a589-c1f0062d4f95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:16.512321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09b87d-4cab-4af8-a589-c1f0062d4f95 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:16.540492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1155/2314] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:16 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:16.900369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fa874-e4a2-4e57-bc58-1d4b49c3955f service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:16.903035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-541fa874-e4a2-4e57-bc58-1d4b49c3955f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:16.937957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fa874-e4a2-4e57-bc58-1d4b49c3955f service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:16.945917 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:16.963519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:16.963519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:16.963519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] PATH_INFO: `/v3/projects/8cf171d42bd64851869a81c2a0c82ddf` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:16.963519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cf171d42bd64851869a81c2a0c82ddf)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:16.963519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:17.164544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445268c8-05c7-4445-84a1-0a551cfb82de None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:17.165363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445268c8-05c7-4445-84a1-0a551cfb82de None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:17.165363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445268c8-05c7-4445-84a1-0a551cfb82de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:17.195725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-445268c8-05c7-4445-84a1-0a551cfb82de None None] MFA Rules not processed for user `1658e3b405ea41939ff42b6bb36d87f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:17.197344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-445268c8-05c7-4445-84a1-0a551cfb82de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:17.294632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1156/2315] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:17.329526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cf171d42bd64851869a81c2a0c82ddf', 'request_id': 'req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:17.338323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cf171d42bd64851869a81c2a0c82ddf', 'request_id': 'req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:17.472087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cf171d42bd64851869a81c2a0c82ddf', 'request_id': 'req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:17.489780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] The token cache is being invalidate because project 8cf171d42bd64851869a81c2a0c82ddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:32:17.490117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:17.491062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:17.492976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1160/2316] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:32:16 2022] DELETE /identity/v3/projects/8cf171d42bd64851869a81c2a0c82ddf => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:17.498035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445268c8-05c7-4445-84a1-0a551cfb82de None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:17.510004 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445268c8-05c7-4445-84a1-0a551cfb82de None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:17.627304 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:17.628341 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:17.628889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:17.629365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] PATH_INFO: `/v3/users/c7472a113f494acfaac58182a21fc422` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:17.630832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7472a113f494acfaac58182a21fc422)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:17.633003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:17.864803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7472a113f494acfaac58182a21fc422'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:17.949256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7472a113f494acfaac58182a21fc422'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:17.950987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7472a113f494acfaac58182a21fc422'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:17.974603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7472a113f494acfaac58182a21fc422'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:17.975941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c7f92ecc-c5d3-4f10-aae6-2f92dc5e5d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7472a113f494acfaac58182a21fc422'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:18.005159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1157/2317] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:32:17 2022] DELETE /identity/v3/users/c7472a113f494acfaac58182a21fc422 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:18.457005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:18.793382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:18.804073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:18.806195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab33f7-2203-4887-bc2a-27665dfc1dde None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:18.806693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab33f7-2203-4887-bc2a-27665dfc1dde None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:18.811313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab33f7-2203-4887-bc2a-27665dfc1dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:18.836370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e9a22b-bf6b-4c15-8f34-51f01380d3aa admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:18.876480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e2ab33f7-2203-4887-bc2a-27665dfc1dde None None] MFA Rules not processed for user `da9978cab3fb4bdda39a2e4718ee2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:18.879012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e2ab33f7-2203-4887-bc2a-27665dfc1dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:18.947266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:18.948347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:18.948817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:18.949266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] PATH_INFO: `/v3/projects/fabdc17fa59b4ab0a1b36bda87831cfb` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:18.951944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fabdc17fa59b4ab0a1b36bda87831cfb)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:18.961959 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:18.995828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1158/2318] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:19.225248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fabdc17fa59b4ab0a1b36bda87831cfb', 'request_id': 'req-8e90435d-ce1c-4529-9b34-f460cc06317d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:19.240400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fabdc17fa59b4ab0a1b36bda87831cfb', 'request_id': 'req-8e90435d-ce1c-4529-9b34-f460cc06317d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:19.327127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fabdc17fa59b4ab0a1b36bda87831cfb', 'request_id': 'req-8e90435d-ce1c-4529-9b34-f460cc06317d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:19.363419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] The token cache is being invalidate because project fabdc17fa59b4ab0a1b36bda87831cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:32:19.365331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:19.366537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:19.370290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1161/2319] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:32:18 2022] DELETE /identity/v3/projects/fabdc17fa59b4ab0a1b36bda87831cfb => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:20.335235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:21.206453 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c903d0-1bf1-487c-874a-012c8739d76b None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:21.208514 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c903d0-1bf1-487c-874a-012c8739d76b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:21.210802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c903d0-1bf1-487c-874a-012c8739d76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:21.240421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-51c903d0-1bf1-487c-874a-012c8739d76b None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:21.248265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-51c903d0-1bf1-487c-874a-012c8739d76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:21.406283 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1159/2320] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:32:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:21.518761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:21.521099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:21.640901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90435d-ce1c-4529-9b34-f460cc06317d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:21.647739 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197111d2-7cca-447d-8b79-5266be341ce1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:21.648633 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197111d2-7cca-447d-8b79-5266be341ce1 service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:21.649072 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197111d2-7cca-447d-8b79-5266be341ce1 service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:21.649440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197111d2-7cca-447d-8b79-5266be341ce1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:21.660548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197111d2-7cca-447d-8b79-5266be341ce1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:21.662247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197111d2-7cca-447d-8b79-5266be341ce1 service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:21.672400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1162/2321] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:32:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 01:32:22.598402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:25.092974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:25.535353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:28.604443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:28.943540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:30.823891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:32.477265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:34.873277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:37.647529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:38.136369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:41.022907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:41.247603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:42.331746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:42.943285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:43.481960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e9b36-67db-47d2-b1d9-7da479f7d9f6 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:43.482246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e9b36-67db-47d2-b1d9-7da479f7d9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:43.490075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e9b36-67db-47d2-b1d9-7da479f7d9f6 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:43.490075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1160/2322] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:32:43.505550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2ca57-91c1-4b74-9b43-61f0d838711d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:43.512489 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2ca57-91c1-4b74-9b43-61f0d838711d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:43.512489 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2ca57-91c1-4b74-9b43-61f0d838711d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:43.542594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-28a2ca57-91c1-4b74-9b43-61f0d838711d None None] MFA Rules not processed for user `9f73bca3896440258430cfce042faa42`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:43.548001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-28a2ca57-91c1-4b74-9b43-61f0d838711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:43.674917 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1163/2323] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:32:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:32:43.687814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e9b36-67db-47d2-b1d9-7da479f7d9f6 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:43.689639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ab8e9b36-67db-47d2-b1d9-7da479f7d9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:43.709615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2ca57-91c1-4b74-9b43-61f0d838711d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:43.711843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-28a2ca57-91c1-4b74-9b43-61f0d838711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:43.717201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e9b36-67db-47d2-b1d9-7da479f7d9f6 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:43.741270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d85e8d-7ced-4116-bfe1-22acb0b80601 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:43.742050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d85e8d-7ced-4116-bfe1-22acb0b80601 service placement] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:43.742327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d85e8d-7ced-4116-bfe1-22acb0b80601 service placement] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:43.742603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d85e8d-7ced-4116-bfe1-22acb0b80601 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:43.755300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d85e8d-7ced-4116-bfe1-22acb0b80601 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:43.757693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d85e8d-7ced-4116-bfe1-22acb0b80601 service placement] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:43.759981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:43.769683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1161/2324] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:43.828429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2ca57-91c1-4b74-9b43-61f0d838711d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:43.843860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:43.849318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:43.850113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:43.850740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] PATH_INFO: `/v3/users/96f28349adf24ae7acfa9d02fddfcf98` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:43.853080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f28349adf24ae7acfa9d02fddfcf98)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:43.858331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:44.208520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f28349adf24ae7acfa9d02fddfcf98'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:44.208520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f28349adf24ae7acfa9d02fddfcf98'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:44.208520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f28349adf24ae7acfa9d02fddfcf98'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:44.220382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f28349adf24ae7acfa9d02fddfcf98'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:44.251070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f28349adf24ae7acfa9d02fddfcf98'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:44.356971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1164/2325] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:32:43 2022] DELETE /identity/v3/users/96f28349adf24ae7acfa9d02fddfcf98 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:44.732886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d85e8d-7ced-4116-bfe1-22acb0b80601 service placement] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:44.751827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d85e8d-7ced-4116-bfe1-22acb0b80601 service placement] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:44.910046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:44.910755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:44.911040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:44.911326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] PATH_INFO: `/v3/projects/4240f3dbafbb4d8abbf79d6e8a2648dd` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:44.912912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4240f3dbafbb4d8abbf79d6e8a2648dd)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:44.914896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:45.124199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4240f3dbafbb4d8abbf79d6e8a2648dd', 'request_id': 'req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:45.133642 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4240f3dbafbb4d8abbf79d6e8a2648dd', 'request_id': 'req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:45.285091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4240f3dbafbb4d8abbf79d6e8a2648dd', 'request_id': 'req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:45.308241 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] The token cache is being invalidate because project 4240f3dbafbb4d8abbf79d6e8a2648dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:32:45.308743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:45.310596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:45.313699 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1162/2326] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:32:44 2022] DELETE /identity/v3/projects/4240f3dbafbb4d8abbf79d6e8a2648dd => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:45.326136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:45.326956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:45.329158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:45.359001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503b47d-5aa9-46fa-9c59-a38c4a0308e4 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:45.472255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:45.473198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:45.473655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:45.474081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] PATH_INFO: `/v3/users/d0a24e20d5ec48cfb5936380e0914a7d` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:45.476833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a24e20d5ec48cfb5936380e0914a7d)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:45.480037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:45.686456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:45.691643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:45.715794 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:45.725969 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0883f4-93b4-4df9-88dd-f9b8b0338d54 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:45.727713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a24e20d5ec48cfb5936380e0914a7d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:45.732073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a24e20d5ec48cfb5936380e0914a7d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:45.736586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a24e20d5ec48cfb5936380e0914a7d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:45.738122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:45.739659 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:45.740169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:45.740664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] PATH_INFO: `/v3/users/ce82595e190e40268a8788a2de45e56c` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:45.743921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce82595e190e40268a8788a2de45e56c)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:45.750833 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:45.756508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a24e20d5ec48cfb5936380e0914a7d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:45.783907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a24e20d5ec48cfb5936380e0914a7d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:45.933631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1165/2327] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:32:45 2022] DELETE /identity/v3/users/d0a24e20d5ec48cfb5936380e0914a7d => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:46.240177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce82595e190e40268a8788a2de45e56c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:46.300346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:46.303408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:46.324930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce82595e190e40268a8788a2de45e56c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:46.326220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce82595e190e40268a8788a2de45e56c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:46.393538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce82595e190e40268a8788a2de45e56c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:46.395408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-06bcd7ea-40dc-4833-9c61-d72b2f96d450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce82595e190e40268a8788a2de45e56c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:46.427617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1163/2328] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:32:45 2022] DELETE /identity/v3/users/ce82595e190e40268a8788a2de45e56c => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:32:46.435988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc832b-2815-4912-b2e3-35a742218736 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:46.436502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc832b-2815-4912-b2e3-35a742218736 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:46.437000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc832b-2815-4912-b2e3-35a742218736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:46.480524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-71dc832b-2815-4912-b2e3-35a742218736 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:46.484323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-71dc832b-2815-4912-b2e3-35a742218736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:46.493387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3fe61-b310-4a8c-b06d-723c44d7cd8b admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:46.594620 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:46.595637 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:46.596206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:46.596769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] PATH_INFO: `/v3/projects/97978ed6ec814ea89cbf6721409cfd00` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:46.600326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1164/2329] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:46.600521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97978ed6ec814ea89cbf6721409cfd00)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:46.604351 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:46.611697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc832b-2815-4912-b2e3-35a742218736 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:46.614097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-71dc832b-2815-4912-b2e3-35a742218736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:46.710910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc832b-2815-4912-b2e3-35a742218736 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:46.716158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db440cf5-84fb-4ddb-9099-50956fce1127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:46.716870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db440cf5-84fb-4ddb-9099-50956fce1127 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:46.717243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db440cf5-84fb-4ddb-9099-50956fce1127 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:46.717830 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db440cf5-84fb-4ddb-9099-50956fce1127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:46.731602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db440cf5-84fb-4ddb-9099-50956fce1127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:46.733842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db440cf5-84fb-4ddb-9099-50956fce1127 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:46.749467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1165/2330] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:46.825895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97978ed6ec814ea89cbf6721409cfd00', 'request_id': 'req-7554f2ee-271a-4fae-a50a-4688386feff8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:46.849376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97978ed6ec814ea89cbf6721409cfd00', 'request_id': 'req-7554f2ee-271a-4fae-a50a-4688386feff8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:46.857702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:46.963285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97978ed6ec814ea89cbf6721409cfd00', 'request_id': 'req-7554f2ee-271a-4fae-a50a-4688386feff8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:46.984705 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] The token cache is being invalidate because project 97978ed6ec814ea89cbf6721409cfd00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:32:46.985014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:46.985831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-7554f2ee-271a-4fae-a50a-4688386feff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:46.988898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1166/2331] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:32:46 2022] DELETE /identity/v3/projects/97978ed6ec814ea89cbf6721409cfd00 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:32:46.996691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538efd-ee09-4c75-ba91-63182cffc7d4 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:46.997226 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538efd-ee09-4c75-ba91-63182cffc7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:46.997741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2538efd-ee09-4c75-ba91-63182cffc7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:47.029057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d2538efd-ee09-4c75-ba91-63182cffc7d4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:47.032066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d2538efd-ee09-4c75-ba91-63182cffc7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:47.074595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:47.196685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1166/2332] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:47.202345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be4eb1-b8d6-4299-8a3b-3afb39b1c71a None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:47.202844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be4eb1-b8d6-4299-8a3b-3afb39b1c71a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:47.203330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be4eb1-b8d6-4299-8a3b-3afb39b1c71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:47.250434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-30be4eb1-b8d6-4299-8a3b-3afb39b1c71a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:47.252229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-30be4eb1-b8d6-4299-8a3b-3afb39b1c71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:47.269435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1167/2333] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:47.271377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2538efd-ee09-4c75-ba91-63182cffc7d4 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:47.294407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2538efd-ee09-4c75-ba91-63182cffc7d4 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:47.319106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f54f-9d80-47c7-8dae-a9bd726faa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:47.320006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f54f-9d80-47c7-8dae-a9bd726faa4a admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:47.325754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f54f-9d80-47c7-8dae-a9bd726faa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:47.326178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f54f-9d80-47c7-8dae-a9bd726faa4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:47.327393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971f54f-9d80-47c7-8dae-a9bd726faa4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:32:47.330374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971f54f-9d80-47c7-8dae-a9bd726faa4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:47.333334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971f54f-9d80-47c7-8dae-a9bd726faa4a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:47.357481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1167/2334] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:32:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:32:47.365117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be4eb1-b8d6-4299-8a3b-3afb39b1c71a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:47.382906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be4eb1-b8d6-4299-8a3b-3afb39b1c71a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:47.397659 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadc458-7ae2-4548-ae23-e419fb169813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:47.398667 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc458-7ae2-4548-ae23-e419fb169813 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:47.399745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc458-7ae2-4548-ae23-e419fb169813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:47.400133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc458-7ae2-4548-ae23-e419fb169813 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:47.401505 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadc458-7ae2-4548-ae23-e419fb169813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:47.405207 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadc458-7ae2-4548-ae23-e419fb169813 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:47.436350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:47.540066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1168/2335] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:32:47 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:32:47.544987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f54f-9d80-47c7-8dae-a9bd726faa4a admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:47.557838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f54f-9d80-47c7-8dae-a9bd726faa4a admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:47.646794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff7302-a458-4db6-9922-df0db8559bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:47.648053 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff7302-a458-4db6-9922-df0db8559bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:47.648695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff7302-a458-4db6-9922-df0db8559bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:47.649068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff7302-a458-4db6-9922-df0db8559bfb admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:47.649907 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff7302-a458-4db6-9922-df0db8559bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:47.652281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff7302-a458-4db6-9922-df0db8559bfb admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:47.679922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadc458-7ae2-4548-ae23-e419fb169813 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:47.683444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-adadc458-7ae2-4548-ae23-e419fb169813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:47.707583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadc458-7ae2-4548-ae23-e419fb169813 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:47.714505 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:47.715466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:47.715982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:47.717169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] PATH_INFO: `/v3/projects/5a1bb8bc9a6542469dc619216bf570e8` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:47.718957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1bb8bc9a6542469dc619216bf570e8)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:47.722080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:47.743396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1168/2336] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:32:47 2022] POST /identity/v3/users => generated 292 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:32:47.753467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff7302-a458-4db6-9922-df0db8559bfb admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:47.771314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff7302-a458-4db6-9922-df0db8559bfb admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:47.779311 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5aafd5-47e0-45e7-afbc-c7502aa01447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:47.780152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aafd5-47e0-45e7-afbc-c7502aa01447 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:47.780528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aafd5-47e0-45e7-afbc-c7502aa01447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:47.780922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5aafd5-47e0-45e7-afbc-c7502aa01447 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:47.781770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5aafd5-47e0-45e7-afbc-c7502aa01447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:47.783721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5aafd5-47e0-45e7-afbc-c7502aa01447 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:47.801726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1169/2337] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:32:47.806232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5aafd5-47e0-45e7-afbc-c7502aa01447 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:47.826806 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5aafd5-47e0-45e7-afbc-c7502aa01447 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:47.834621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ffba5-f681-4e19-bb7c-aa6b3e3cb8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:47.835179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ffba5-f681-4e19-bb7c-aa6b3e3cb8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:47.835486 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ffba5-f681-4e19-bb7c-aa6b3e3cb8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:47.841565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ffba5-f681-4e19-bb7c-aa6b3e3cb8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:47.841565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ffba5-f681-4e19-bb7c-aa6b3e3cb8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:47.841565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ffba5-f681-4e19-bb7c-aa6b3e3cb8a4 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:47.850575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1170/2338] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:32:47.857117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ffba5-f681-4e19-bb7c-aa6b3e3cb8a4 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:47.868331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ffba5-f681-4e19-bb7c-aa6b3e3cb8a4 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:47.874956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bbe5e-bde6-485d-a25b-f5f09cb99ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:47.875887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bbe5e-bde6-485d-a25b-f5f09cb99ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:47.876091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bbe5e-bde6-485d-a25b-f5f09cb99ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:47.876373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bbe5e-bde6-485d-a25b-f5f09cb99ed7 admin admin] PATH_INFO: `/v3/projects/a9e1b329442b49838334be2e4146fef9/users/6f512580b5fd41c0a701899232ce79c8/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:47.899626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bbe5e-bde6-485d-a25b-f5f09cb99ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e1b329442b49838334be2e4146fef9, user_id=6f512580b5fd41c0a701899232ce79c8, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:47.903495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bbe5e-bde6-485d-a25b-f5f09cb99ed7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:47.908189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1bb8bc9a6542469dc619216bf570e8', 'request_id': 'req-3891222d-8504-4477-be8d-42deae927213'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:47.920943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1bb8bc9a6542469dc619216bf570e8', 'request_id': 'req-3891222d-8504-4477-be8d-42deae927213'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:47.963844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:48.030740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1171/2339] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:47 2022] PUT /identity/v3/projects/a9e1b329442b49838334be2e4146fef9/users/6f512580b5fd41c0a701899232ce79c8/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:32:48.038042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1bb8bc9a6542469dc619216bf570e8', 'request_id': 'req-3891222d-8504-4477-be8d-42deae927213'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:48.045840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ad819-44d1-43a8-ba7a-9445e3b7616f None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:48.045840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ad819-44d1-43a8-ba7a-9445e3b7616f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:48.045840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ad819-44d1-43a8-ba7a-9445e3b7616f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:48.060937 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] The token cache is being invalidate because project 5a1bb8bc9a6542469dc619216bf570e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:32:48.061537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:48.062963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3891222d-8504-4477-be8d-42deae927213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:32:48.065887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1169/2340] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:32:47 2022] DELETE /identity/v3/projects/5a1bb8bc9a6542469dc619216bf570e8 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:48.072565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb5759-3703-4f0c-b1ad-fd6b4d5c7cc9 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:48.073041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb5759-3703-4f0c-b1ad-fd6b4d5c7cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:48.073526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb5759-3703-4f0c-b1ad-fd6b4d5c7cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:48.084343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ab1ad819-44d1-43a8-ba7a-9445e3b7616f None None] MFA Rules not processed for user `6f512580b5fd41c0a701899232ce79c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:48.086216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ab1ad819-44d1-43a8-ba7a-9445e3b7616f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:48.099879 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-94cb5759-3703-4f0c-b1ad-fd6b4d5c7cc9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:48.101180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-94cb5759-3703-4f0c-b1ad-fd6b4d5c7cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:48.194832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1172/2341] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:48.200176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ad819-44d1-43a8-ba7a-9445e3b7616f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:48.206192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1170/2342] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:32:48.206449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ab1ad819-44d1-43a8-ba7a-9445e3b7616f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:48.214177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5a23b-e620-4db7-bea8-c4bd1be33cb6 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:48.214740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5a23b-e620-4db7-bea8-c4bd1be33cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:48.217360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5a23b-e620-4db7-bea8-c4bd1be33cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:48.226101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ad819-44d1-43a8-ba7a-9445e3b7616f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:48.241805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cd16f-ffea-4c92-bf19-4fb4d3b59d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:48.243606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cd16f-ffea-4c92-bf19-4fb4d3b59d3f admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:48.243606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cd16f-ffea-4c92-bf19-4fb4d3b59d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:48.243931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cd16f-ffea-4c92-bf19-4fb4d3b59d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:48.244097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c0f5a23b-e620-4db7-bea8-c4bd1be33cb6 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:48.244503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576cd16f-ffea-4c92-bf19-4fb4d3b59d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:48.245638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c0f5a23b-e620-4db7-bea8-c4bd1be33cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:48.247974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576cd16f-ffea-4c92-bf19-4fb4d3b59d3f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:48.255863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1171/2343] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:48.261693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5a23b-e620-4db7-bea8-c4bd1be33cb6 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:48.283158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5a23b-e620-4db7-bea8-c4bd1be33cb6 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:48.295989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0c645-c2bb-49a6-9494-c083b1896cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:48.296762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0c645-c2bb-49a6-9494-c083b1896cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:48.297013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0c645-c2bb-49a6-9494-c083b1896cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:48.297365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0c645-c2bb-49a6-9494-c083b1896cd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:48.298486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0c645-c2bb-49a6-9494-c083b1896cd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:32:48.298871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0c645-c2bb-49a6-9494-c083b1896cd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:48.300787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0c645-c2bb-49a6-9494-c083b1896cd2 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:48.327202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1172/2344] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:32:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:32:48.336416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0c645-c2bb-49a6-9494-c083b1896cd2 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:48.357032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0c645-c2bb-49a6-9494-c083b1896cd2 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:48.365015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6bef77-0d4f-43b9-befa-9757309fdcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:48.366068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bef77-0d4f-43b9-befa-9757309fdcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:48.366563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bef77-0d4f-43b9-befa-9757309fdcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:48.367310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bef77-0d4f-43b9-befa-9757309fdcd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:48.368459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6bef77-0d4f-43b9-befa-9757309fdcd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:48.371086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6bef77-0d4f-43b9-befa-9757309fdcd4 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:48.404344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1173/2345] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:32:48 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:32:48.407981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cd16f-ffea-4c92-bf19-4fb4d3b59d3f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:48.427951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:48.524495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1173/2346] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:32:48 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:32:48.527506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6bef77-0d4f-43b9-befa-9757309fdcd4 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:48.549394 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cd16f-ffea-4c92-bf19-4fb4d3b59d3f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:48.685252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f77fb5-7499-440d-b887-5813ce212928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:48.686228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f77fb5-7499-440d-b887-5813ce212928 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:48.686799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f77fb5-7499-440d-b887-5813ce212928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:48.687309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f77fb5-7499-440d-b887-5813ce212928 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:48.688299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f77fb5-7499-440d-b887-5813ce212928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:48.690796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f77fb5-7499-440d-b887-5813ce212928 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:48.740798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6bef77-0d4f-43b9-befa-9757309fdcd4 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:48.750747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06eaf9-6177-47c2-ab7c-17801427f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:48.751872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06eaf9-6177-47c2-ab7c-17801427f63c admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:48.752308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06eaf9-6177-47c2-ab7c-17801427f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:48.752848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06eaf9-6177-47c2-ab7c-17801427f63c admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:48.754090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06eaf9-6177-47c2-ab7c-17801427f63c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:48.758523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06eaf9-6177-47c2-ab7c-17801427f63c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:48.770234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:48.835458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1174/2347] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:32:48 2022] POST /identity/v3/users => generated 299 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:32:48.842397 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f77fb5-7499-440d-b887-5813ce212928 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:48.863359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f77fb5-7499-440d-b887-5813ce212928 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:48.873177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31348-a358-44aa-9dad-690f900fb1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:48.882675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31348-a358-44aa-9dad-690f900fb1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:48.882675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31348-a358-44aa-9dad-690f900fb1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:48.882675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c31348-a358-44aa-9dad-690f900fb1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:48.882675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c31348-a358-44aa-9dad-690f900fb1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:48.882675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c31348-a358-44aa-9dad-690f900fb1f3 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:48.903846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1175/2348] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:32:48.905578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31348-a358-44aa-9dad-690f900fb1f3 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:48.921057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c31348-a358-44aa-9dad-690f900fb1f3 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:48.929422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c17969-b41e-41d1-aba9-5e15fc87796b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:48.939896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c17969-b41e-41d1-aba9-5e15fc87796b admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:48.939896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c17969-b41e-41d1-aba9-5e15fc87796b admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:48.939896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c17969-b41e-41d1-aba9-5e15fc87796b admin admin] PATH_INFO: `/v3/projects/d4d7817355bf4677803eff2b0ef4df6b/users/4cc62aa12c3644aaa1c16c912e1806b3/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:48.962296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c17969-b41e-41d1-aba9-5e15fc87796b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d7817355bf4677803eff2b0ef4df6b, user_id=4cc62aa12c3644aaa1c16c912e1806b3, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:48.969006 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c17969-b41e-41d1-aba9-5e15fc87796b admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:48.993808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1174/2349] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:32:48 2022] POST /identity/v3/users => generated 288 bytes in 468 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:32:48.999998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06eaf9-6177-47c2-ab7c-17801427f63c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:49.021554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06eaf9-6177-47c2-ab7c-17801427f63c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:49.031050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb40d1-f3e4-4f09-97a2-44427602f804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:49.032398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb40d1-f3e4-4f09-97a2-44427602f804 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:49.032905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb40d1-f3e4-4f09-97a2-44427602f804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:49.033390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb40d1-f3e4-4f09-97a2-44427602f804 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:49.035119 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb40d1-f3e4-4f09-97a2-44427602f804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:49.038517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb40d1-f3e4-4f09-97a2-44427602f804 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:49.071605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1175/2350] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:32:49.076733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb40d1-f3e4-4f09-97a2-44427602f804 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:49.100952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb40d1-f3e4-4f09-97a2-44427602f804 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:49.113474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1faa3-acab-45ca-ab62-d4dd1c554033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:49.115306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1faa3-acab-45ca-ab62-d4dd1c554033 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:49.115306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1faa3-acab-45ca-ab62-d4dd1c554033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:49.115595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1faa3-acab-45ca-ab62-d4dd1c554033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:49.116722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1faa3-acab-45ca-ab62-d4dd1c554033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:49.119877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1faa3-acab-45ca-ab62-d4dd1c554033 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:49.140330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1176/2351] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:32:49.147010 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1faa3-acab-45ca-ab62-d4dd1c554033 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:49.152038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1176/2352] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:48 2022] PUT /identity/v3/projects/d4d7817355bf4677803eff2b0ef4df6b/users/4cc62aa12c3644aaa1c16c912e1806b3/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:49.156978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c17969-b41e-41d1-aba9-5e15fc87796b admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:49.165657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1faa3-acab-45ca-ab62-d4dd1c554033 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:49.281026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07319326-8fdd-43d6-9a10-1b76425ff7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:49.282275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07319326-8fdd-43d6-9a10-1b76425ff7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:49.282625 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07319326-8fdd-43d6-9a10-1b76425ff7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:49.283203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07319326-8fdd-43d6-9a10-1b76425ff7c7 admin admin] PATH_INFO: `/v3/projects/84a9bb6a15444802af12e048ea53223d/users/69e81cfb7aa24918b5ec8b32064e17eb/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:49.328233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c17969-b41e-41d1-aba9-5e15fc87796b admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:49.334840 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07319326-8fdd-43d6-9a10-1b76425ff7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a9bb6a15444802af12e048ea53223d, user_id=69e81cfb7aa24918b5ec8b32064e17eb, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:49.340029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07319326-8fdd-43d6-9a10-1b76425ff7c7 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:49.341748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf977f8-01be-487d-93a4-0270cc1d4484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:49.342761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf977f8-01be-487d-93a4-0270cc1d4484 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:49.343417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf977f8-01be-487d-93a4-0270cc1d4484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:49.343913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf977f8-01be-487d-93a4-0270cc1d4484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:49.344836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf977f8-01be-487d-93a4-0270cc1d4484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:49.347488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf977f8-01be-487d-93a4-0270cc1d4484 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:49.369798 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1177/2353] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:32:49.374883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf977f8-01be-487d-93a4-0270cc1d4484 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:49.404487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf977f8-01be-487d-93a4-0270cc1d4484 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:49.415786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b71951-6b97-409d-bbe6-4a36c9452f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:49.417073 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b71951-6b97-409d-bbe6-4a36c9452f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:49.417584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b71951-6b97-409d-bbe6-4a36c9452f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:49.418284 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b71951-6b97-409d-bbe6-4a36c9452f84 admin admin] PATH_INFO: `/v3/projects/d4d7817355bf4677803eff2b0ef4df6b/users/4cc62aa12c3644aaa1c16c912e1806b3/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:49.421869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b71951-6b97-409d-bbe6-4a36c9452f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4d7817355bf4677803eff2b0ef4df6b, user_id=4cc62aa12c3644aaa1c16c912e1806b3, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:49.427260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b71951-6b97-409d-bbe6-4a36c9452f84 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:49.452451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1178/2354] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:49 2022] PUT /identity/v3/projects/d4d7817355bf4677803eff2b0ef4df6b/users/4cc62aa12c3644aaa1c16c912e1806b3/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:49.457594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8c07f-671c-4a42-a96b-c9febacc226d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:49.458799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8c07f-671c-4a42-a96b-c9febacc226d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:49.460019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db8c07f-671c-4a42-a96b-c9febacc226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:49.510334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1177/2355] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:32:49 2022] PUT /identity/v3/projects/84a9bb6a15444802af12e048ea53223d/users/69e81cfb7aa24918b5ec8b32064e17eb/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:32:49.571731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-0db8c07f-671c-4a42-a96b-c9febacc226d None None] MFA Rules not processed for user `4cc62aa12c3644aaa1c16c912e1806b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:49.576235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0db8c07f-671c-4a42-a96b-c9febacc226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:49.612394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe49723-ef3b-43b8-9a4f-471fabfcc988 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:49.612978 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe49723-ef3b-43b8-9a4f-471fabfcc988 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:49.613486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe49723-ef3b-43b8-9a4f-471fabfcc988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:49.659814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-4fe49723-ef3b-43b8-9a4f-471fabfcc988 None None] MFA Rules not processed for user `69e81cfb7aa24918b5ec8b32064e17eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:49.676259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4fe49723-ef3b-43b8-9a4f-471fabfcc988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:49.726033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:49.770429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1179/2356] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:49.795097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8c07f-671c-4a42-a96b-c9febacc226d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:49.796930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0db8c07f-671c-4a42-a96b-c9febacc226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:49.852346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1178/2357] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:32:49.866230 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe49723-ef3b-43b8-9a4f-471fabfcc988 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:49.869268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4fe49723-ef3b-43b8-9a4f-471fabfcc988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:49.925188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db8c07f-671c-4a42-a96b-c9febacc226d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:49.932488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e97365-6bee-482a-80f2-7c87a95f442b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:49.933923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97365-6bee-482a-80f2-7c87a95f442b service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:49.934632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97365-6bee-482a-80f2-7c87a95f442b service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:49.935231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97365-6bee-482a-80f2-7c87a95f442b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:50.068628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe49723-ef3b-43b8-9a4f-471fabfcc988 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:50.076287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6944b3-47c6-447d-a45e-53fd8920f17d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:50.077481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6944b3-47c6-447d-a45e-53fd8920f17d service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:50.077987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6944b3-47c6-447d-a45e-53fd8920f17d service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:50.078473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6944b3-47c6-447d-a45e-53fd8920f17d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:50.117690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e97365-6bee-482a-80f2-7c87a95f442b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:50.119694 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6944b3-47c6-447d-a45e-53fd8920f17d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:50.123016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e97365-6bee-482a-80f2-7c87a95f442b service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:50.123809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6944b3-47c6-447d-a45e-53fd8920f17d service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:50.150829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1180/2358] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:32:50.156518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1179/2359] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:50.438442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:51.671817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:52.500157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212afc5-63a6-4475-be06-a8f2afb9445c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:52.500911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212afc5-63a6-4475-be06-a8f2afb9445c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:52.500911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212afc5-63a6-4475-be06-a8f2afb9445c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:52.522998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b212afc5-63a6-4475-be06-a8f2afb9445c None None] MFA Rules not processed for user `69e81cfb7aa24918b5ec8b32064e17eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:52.525236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b212afc5-63a6-4475-be06-a8f2afb9445c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:52.534253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1181/2360] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:32:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:52.577492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:53.022940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6944b3-47c6-447d-a45e-53fd8920f17d service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:53.027038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-fd6944b3-47c6-447d-a45e-53fd8920f17d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:53.151809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6944b3-47c6-447d-a45e-53fd8920f17d service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:53.157752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2940d-55da-4c6b-a0be-0eca725f113a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:53.158390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2940d-55da-4c6b-a0be-0eca725f113a service placement] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:53.158680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2940d-55da-4c6b-a0be-0eca725f113a service placement] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:53.158952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2940d-55da-4c6b-a0be-0eca725f113a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:53.160932 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-38e2940d-55da-4c6b-a0be-0eca725f113a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:53.178490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2940d-55da-4c6b-a0be-0eca725f113a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:53.181666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2940d-55da-4c6b-a0be-0eca725f113a service placement] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:53.203094 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1180/2361] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:53.301166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:53.771226 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:53.866404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:54.313982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:54.579478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a154d-3be8-496e-b810-2ab1eacedbe2 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:54.579966 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a154d-3be8-496e-b810-2ab1eacedbe2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:54.580831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a154d-3be8-496e-b810-2ab1eacedbe2 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:54.583835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1182/2362] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:32:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:32:54.594622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799c18d-3841-4fa1-b94d-99169efde5af None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:54.595136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799c18d-3841-4fa1-b94d-99169efde5af None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:54.595439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799c18d-3841-4fa1-b94d-99169efde5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:54.624592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-e799c18d-3841-4fa1-b94d-99169efde5af None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:54.625873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e799c18d-3841-4fa1-b94d-99169efde5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:54.659451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:54.756675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1181/2363] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:32:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:54.774551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a154d-3be8-496e-b810-2ab1eacedbe2 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:54.790012 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-468a154d-3be8-496e-b810-2ab1eacedbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:54.918264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a154d-3be8-496e-b810-2ab1eacedbe2 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:54.928831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d243ec-9828-4566-87d8-1cae7b6782cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f10c258fe5f45ccab1669fa32ed6029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:54.930209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d243ec-9828-4566-87d8-1cae7b6782cf service glance] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:54.931892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d243ec-9828-4566-87d8-1cae7b6782cf service glance] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:54.931892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d243ec-9828-4566-87d8-1cae7b6782cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:54.951506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d243ec-9828-4566-87d8-1cae7b6782cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:54.952953 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d243ec-9828-4566-87d8-1cae7b6782cf service glance] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:54.966194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1183/2364] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:55.118296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:55.266610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cebbf-2a92-4939-b312-33b3fff7202b None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:55.267164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cebbf-2a92-4939-b312-33b3fff7202b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:55.267438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89cebbf-2a92-4939-b312-33b3fff7202b None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:55.268984 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1182/2365] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:32:55.274250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72a4fe-b7cb-4965-872e-c5b9b4e51390 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:55.274535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72a4fe-b7cb-4965-872e-c5b9b4e51390 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:55.274815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72a4fe-b7cb-4965-872e-c5b9b4e51390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:55.298044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-6b72a4fe-b7cb-4965-872e-c5b9b4e51390 None None] MFA Rules not processed for user `3b0a7b4f607d4753bf142e1420bcd987`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:55.299293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6b72a4fe-b7cb-4965-872e-c5b9b4e51390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:55.379726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1184/2366] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:32:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:32:55.393501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cebbf-2a92-4939-b312-33b3fff7202b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:55.395909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f89cebbf-2a92-4939-b312-33b3fff7202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:55.417796 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89cebbf-2a92-4939-b312-33b3fff7202b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:55.423786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fc15e-6e12-449d-94e3-5aae5b164cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:55.424913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fc15e-6e12-449d-94e3-5aae5b164cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:55.425507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fc15e-6e12-449d-94e3-5aae5b164cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:55.425797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fc15e-6e12-449d-94e3-5aae5b164cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:55.436139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fc15e-6e12-449d-94e3-5aae5b164cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:55.438991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fc15e-6e12-449d-94e3-5aae5b164cc5 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:55.453380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1183/2367] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:55.462264 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:55.567922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25862077-93f6-4167-9211-f34a0140f4c4 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:55.568564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25862077-93f6-4167-9211-f34a0140f4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:55.569074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25862077-93f6-4167-9211-f34a0140f4c4 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:55.571762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1185/2368] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:32:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:32:55.581455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1c2bd-83c5-44f4-90a3-40260d698dca None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:55.581905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1c2bd-83c5-44f4-90a3-40260d698dca None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:55.582405 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1c2bd-83c5-44f4-90a3-40260d698dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:55.615409 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-4bb1c2bd-83c5-44f4-90a3-40260d698dca None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:32:55.616723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4bb1c2bd-83c5-44f4-90a3-40260d698dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:55.631753 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1184/2369] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:32:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:32:55.634308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:55.653356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25862077-93f6-4167-9211-f34a0140f4c4 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:32:55.656678 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-25862077-93f6-4167-9211-f34a0140f4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:32:55.685731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25862077-93f6-4167-9211-f34a0140f4c4 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:32:55.708412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3b739-027d-49ab-aab5-fce72bcf8edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:32:55.708412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b739-027d-49ab-aab5-fce72bcf8edd service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:32:55.708412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b739-027d-49ab-aab5-fce72bcf8edd service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:32:55.708412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b739-027d-49ab-aab5-fce72bcf8edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:32:55.720619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3b739-027d-49ab-aab5-fce72bcf8edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:32:55.722209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3b739-027d-49ab-aab5-fce72bcf8edd service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:32:55.750582 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1186/2370] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:32:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:32:56.508561 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=100 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:56.605042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:57.029118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:58.061928 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:58.125214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:58.410097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:58.473363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:58.632771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:58.662765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:58.729495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:58.754014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:32:58.998550 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:01.088492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22f57-6a7f-4804-85f0-fc0374f68b63 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:01.089887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22f57-6a7f-4804-85f0-fc0374f68b63 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:01.090446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22f57-6a7f-4804-85f0-fc0374f68b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:01.101211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3b739-027d-49ab-aab5-fce72bcf8edd service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:01.124815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3b739-027d-49ab-aab5-fce72bcf8edd service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:01.135210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e493c-d129-41b5-b871-f58ce3dbd3ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:01.136949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e493c-d129-41b5-b871-f58ce3dbd3ab service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:01.143182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-97b22f57-6a7f-4804-85f0-fc0374f68b63 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:01.143182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-97b22f57-6a7f-4804-85f0-fc0374f68b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:01.144018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e493c-d129-41b5-b871-f58ce3dbd3ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:01.144552 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e493c-d129-41b5-b871-f58ce3dbd3ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:01.178205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e493c-d129-41b5-b871-f58ce3dbd3ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:01.181254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e493c-d129-41b5-b871-f58ce3dbd3ab service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:01.234092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1185/2371] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:33:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:33:01.235794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1187/2372] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:33:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:33:01.258089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22f57-6a7f-4804-85f0-fc0374f68b63 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:01.260710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-97b22f57-6a7f-4804-85f0-fc0374f68b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:01.424555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22f57-6a7f-4804-85f0-fc0374f68b63 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:01.433899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0641f-eece-4464-b39d-fb862f5c7242 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:01.435123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0641f-eece-4464-b39d-fb862f5c7242 service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:01.435688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0641f-eece-4464-b39d-fb862f5c7242 service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:01.436327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0641f-eece-4464-b39d-fb862f5c7242 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:01.452632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0641f-eece-4464-b39d-fb862f5c7242 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:01.455927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0641f-eece-4464-b39d-fb862f5c7242 service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:01.474403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1186/2373] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:33:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:33:11.192385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=66 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:11.467027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=83 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:16.692602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bf0bc-5648-4ba6-8dc5-c922858c9d5a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:16.694336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bf0bc-5648-4ba6-8dc5-c922858c9d5a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:16.695151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4bf0bc-5648-4ba6-8dc5-c922858c9d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:16.740795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ab4bf0bc-5648-4ba6-8dc5-c922858c9d5a None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:16.743293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ab4bf0bc-5648-4ba6-8dc5-c922858c9d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:16.767487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1188/2374] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:33:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:16.781369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0641f-eece-4464-b39d-fb862f5c7242 service swift] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:16.804621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0641f-eece-4464-b39d-fb862f5c7242 service swift] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:16.816131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dec996-ea9a-41d6-aef3-57cb4b2a9a76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:16.817835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dec996-ea9a-41d6-aef3-57cb4b2a9a76 service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:16.818345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dec996-ea9a-41d6-aef3-57cb4b2a9a76 service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:16.818862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dec996-ea9a-41d6-aef3-57cb4b2a9a76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:16.837053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dec996-ea9a-41d6-aef3-57cb4b2a9a76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:16.843537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dec996-ea9a-41d6-aef3-57cb4b2a9a76 service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:16.864155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1187/2375] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:33:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:33:17.248387 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bf0bc-5648-4ba6-8dc5-c922858c9d5a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:17.479929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4bf0bc-5648-4ba6-8dc5-c922858c9d5a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:17.489420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:17.490369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:17.490657 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:17.491072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] PATH_INFO: `/v3/users/6f512580b5fd41c0a701899232ce79c8` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:17.494800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f512580b5fd41c0a701899232ce79c8)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:17.498905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:17.838230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f512580b5fd41c0a701899232ce79c8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:17.957586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f512580b5fd41c0a701899232ce79c8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:17.958529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f512580b5fd41c0a701899232ce79c8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:17.996962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f512580b5fd41c0a701899232ce79c8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:18.000895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f512580b5fd41c0a701899232ce79c8'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:18.047246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1189/2376] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:33:17 2022] DELETE /identity/v3/users/6f512580b5fd41c0a701899232ce79c8 => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:33:18.049034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae699f-01f4-4538-8c0e-cec7f3fd5da5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:18.049950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae699f-01f4-4538-8c0e-cec7f3fd5da5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:18.049950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae699f-01f4-4538-8c0e-cec7f3fd5da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:18.114960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c7ae699f-01f4-4538-8c0e-cec7f3fd5da5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:18.119347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c7ae699f-01f4-4538-8c0e-cec7f3fd5da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:18.347105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1188/2377] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:33:18.369117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:18.372290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:18.607814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84d380-9002-4f11-8030-6ed6349177dd admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:18.613263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915d9ce-acb0-49e5-8ef7-47a3d0c4a12f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:18.614013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915d9ce-acb0-49e5-8ef7-47a3d0c4a12f service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:18.614325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915d9ce-acb0-49e5-8ef7-47a3d0c4a12f service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:18.614606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915d9ce-acb0-49e5-8ef7-47a3d0c4a12f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:18.627828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915d9ce-acb0-49e5-8ef7-47a3d0c4a12f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:18.629321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915d9ce-acb0-49e5-8ef7-47a3d0c4a12f service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:18.658661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1190/2378] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:33:20.292275 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae699f-01f4-4538-8c0e-cec7f3fd5da5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:20.293987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c7ae699f-01f4-4538-8c0e-cec7f3fd5da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:20.317172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae699f-01f4-4538-8c0e-cec7f3fd5da5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:20.326318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:20.328679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:20.328679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:20.328679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] PATH_INFO: `/v3/projects/a9e1b329442b49838334be2e4146fef9` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:20.330466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e1b329442b49838334be2e4146fef9)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:20.332920 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:20.548736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e1b329442b49838334be2e4146fef9', 'request_id': 'req-cb91d0b7-f70b-420e-a75d-be33b1d81d67'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:20.561568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e1b329442b49838334be2e4146fef9', 'request_id': 'req-cb91d0b7-f70b-420e-a75d-be33b1d81d67'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:20.711920 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e1b329442b49838334be2e4146fef9', 'request_id': 'req-cb91d0b7-f70b-420e-a75d-be33b1d81d67'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:20.748302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] The token cache is being invalidate because project a9e1b329442b49838334be2e4146fef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:33:20.749270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:20.751114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:20.756699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1189/2379] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:33:20 2022] DELETE /identity/v3/projects/a9e1b329442b49838334be2e4146fef9 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:33:20.760822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915d9ce-acb0-49e5-8ef7-47a3d0c4a12f service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:20.786709 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915d9ce-acb0-49e5-8ef7-47a3d0c4a12f service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:20.842360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:21.089454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:21.089454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:21.089454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:21.089454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] PATH_INFO: `/v3/users/4cc62aa12c3644aaa1c16c912e1806b3` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:21.089959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc62aa12c3644aaa1c16c912e1806b3)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:21.103164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:21.737662 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc62aa12c3644aaa1c16c912e1806b3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:21.890764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc62aa12c3644aaa1c16c912e1806b3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:21.892460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc62aa12c3644aaa1c16c912e1806b3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:21.925632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc62aa12c3644aaa1c16c912e1806b3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:21.928045 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-c321e279-50c9-44f7-b19f-9ddc92445759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc62aa12c3644aaa1c16c912e1806b3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:21.982715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1191/2380] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:33:20 2022] DELETE /identity/v3/users/4cc62aa12c3644aaa1c16c912e1806b3 => generated 0 bytes in 1222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:33:23.636197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:23.637887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:23.663056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91d0b7-f70b-420e-a75d-be33b1d81d67 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:23.892046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:23.893641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:23.894125 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:23.894920 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] PATH_INFO: `/v3/projects/d4d7817355bf4677803eff2b0ef4df6b` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:23.896893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4d7817355bf4677803eff2b0ef4df6b)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:23.900472 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:23.919619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:24.143159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4d7817355bf4677803eff2b0ef4df6b', 'request_id': 'req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:24.162051 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4d7817355bf4677803eff2b0ef4df6b', 'request_id': 'req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:24.282551 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4d7817355bf4677803eff2b0ef4df6b', 'request_id': 'req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:24.304201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] The token cache is being invalidate because project d4d7817355bf4677803eff2b0ef4df6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:33:24.306556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:24.316184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22fc588c-bd1f-455f-b1c6-ed61bfdfd65f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:24.326441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1190/2381] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:33:23 2022] DELETE /identity/v3/projects/d4d7817355bf4677803eff2b0ef4df6b => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:33:24.330363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafabf1-adca-416b-acd7-e7fec7b93020 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:24.330363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafabf1-adca-416b-acd7-e7fec7b93020 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:24.330363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cafabf1-adca-416b-acd7-e7fec7b93020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:24.439406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-4cafabf1-adca-416b-acd7-e7fec7b93020 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:24.441473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4cafabf1-adca-416b-acd7-e7fec7b93020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:24.730986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1192/2382] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:24.741274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6c66a-572b-43c4-9e57-5cd2ccc58f98 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:24.741827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6c66a-572b-43c4-9e57-5cd2ccc58f98 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:24.741925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6c66a-572b-43c4-9e57-5cd2ccc58f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:24.812314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-00f6c66a-572b-43c4-9e57-5cd2ccc58f98 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:24.821519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-00f6c66a-572b-43c4-9e57-5cd2ccc58f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:24.869338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1191/2383] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:24.870740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafabf1-adca-416b-acd7-e7fec7b93020 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:24.901513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cafabf1-adca-416b-acd7-e7fec7b93020 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:24.913913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6ddcf-1e01-4dad-a311-e0740cea521d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:24.914640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6ddcf-1e01-4dad-a311-e0740cea521d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:24.914930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6ddcf-1e01-4dad-a311-e0740cea521d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:24.915265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6ddcf-1e01-4dad-a311-e0740cea521d admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:24.916140 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6ddcf-1e01-4dad-a311-e0740cea521d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:33:24.916494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6ddcf-1e01-4dad-a311-e0740cea521d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:24.921722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6ddcf-1e01-4dad-a311-e0740cea521d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:24.961393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1193/2384] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:33:24.964120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6c66a-572b-43c4-9e57-5cd2ccc58f98 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:24.978527 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6c66a-572b-43c4-9e57-5cd2ccc58f98 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:24.991347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fa8cd-0107-4f3b-b8ca-3d33a4c19cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:24.991347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fa8cd-0107-4f3b-b8ca-3d33a4c19cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:24.991347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fa8cd-0107-4f3b-b8ca-3d33a4c19cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:24.991347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fa8cd-0107-4f3b-b8ca-3d33a4c19cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:24.992532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fa8cd-0107-4f3b-b8ca-3d33a4c19cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:24.992532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6fa8cd-0107-4f3b-b8ca-3d33a4c19cd7 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:25.140135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1192/2385] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:33:24 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:33:25.143866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6ddcf-1e01-4dad-a311-e0740cea521d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:25.174652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6ddcf-1e01-4dad-a311-e0740cea521d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:25.266138 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:25.370946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae4653-7e06-478d-9da5-912951c9a5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:25.373297 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae4653-7e06-478d-9da5-912951c9a5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:25.373696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae4653-7e06-478d-9da5-912951c9a5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:25.373983 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae4653-7e06-478d-9da5-912951c9a5a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:25.374663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae4653-7e06-478d-9da5-912951c9a5a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:25.377989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae4653-7e06-478d-9da5-912951c9a5a2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:25.531701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1194/2386] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:33:25 2022] POST /identity/v3/users => generated 288 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:33:25.533706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fa8cd-0107-4f3b-b8ca-3d33a4c19cd7 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:25.567055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fa8cd-0107-4f3b-b8ca-3d33a4c19cd7 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:25.600408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9daeaf-0832-4821-947b-142f8760b4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:25.601785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9daeaf-0832-4821-947b-142f8760b4ab admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:25.602314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9daeaf-0832-4821-947b-142f8760b4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:25.602849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9daeaf-0832-4821-947b-142f8760b4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:25.604352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9daeaf-0832-4821-947b-142f8760b4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:25.607847 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9daeaf-0832-4821-947b-142f8760b4ab admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:25.647011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1193/2387] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:33:25.652131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae4653-7e06-478d-9da5-912951c9a5a2 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:25.664138 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae4653-7e06-478d-9da5-912951c9a5a2 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:25.670443 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdea582-ac37-459e-85d7-e41a7919cc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:25.670994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdea582-ac37-459e-85d7-e41a7919cc6d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:25.671340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdea582-ac37-459e-85d7-e41a7919cc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:25.671683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdea582-ac37-459e-85d7-e41a7919cc6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:25.672485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdea582-ac37-459e-85d7-e41a7919cc6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:25.674442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdea582-ac37-459e-85d7-e41a7919cc6d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:25.691225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1195/2388] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:33:25.696584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9daeaf-0832-4821-947b-142f8760b4ab admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:25.709470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9daeaf-0832-4821-947b-142f8760b4ab admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:25.717064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1de922-67c3-4d7f-b6d4-a45a1f8128ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:25.720107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1de922-67c3-4d7f-b6d4-a45a1f8128ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:25.720107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1de922-67c3-4d7f-b6d4-a45a1f8128ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:25.720107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1de922-67c3-4d7f-b6d4-a45a1f8128ed admin admin] PATH_INFO: `/v3/projects/9892b43bd65b4e9f9b86b14a68ed8749/users/0960a67b4dd94c8ba1c382cd9e4adfea/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:25.767440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1de922-67c3-4d7f-b6d4-a45a1f8128ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=9892b43bd65b4e9f9b86b14a68ed8749, user_id=0960a67b4dd94c8ba1c382cd9e4adfea, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:25.775716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1de922-67c3-4d7f-b6d4-a45a1f8128ed admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:25.909891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1194/2389] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:33:25 2022] PUT /identity/v3/projects/9892b43bd65b4e9f9b86b14a68ed8749/users/0960a67b4dd94c8ba1c382cd9e4adfea/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:33:25.943738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19211d-e1f9-4348-bf59-9da9bf0fdf4c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:25.944145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19211d-e1f9-4348-bf59-9da9bf0fdf4c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:25.944801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d19211d-e1f9-4348-bf59-9da9bf0fdf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:26.109319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-4d19211d-e1f9-4348-bf59-9da9bf0fdf4c None None] MFA Rules not processed for user `0960a67b4dd94c8ba1c382cd9e4adfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:26.113222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4d19211d-e1f9-4348-bf59-9da9bf0fdf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:26.396978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1196/2390] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:28.187495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1de922-67c3-4d7f-b6d4-a45a1f8128ed admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:28.192009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ac1de922-67c3-4d7f-b6d4-a45a1f8128ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:28.328346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1de922-67c3-4d7f-b6d4-a45a1f8128ed admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:28.334999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93610c98-1780-4243-b33b-1c153c9c7d73 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f10c258fe5f45ccab1669fa32ed6029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:28.335496 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93610c98-1780-4243-b33b-1c153c9c7d73 service glance] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:28.335919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93610c98-1780-4243-b33b-1c153c9c7d73 service glance] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:28.336471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93610c98-1780-4243-b33b-1c153c9c7d73 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:28.365049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93610c98-1780-4243-b33b-1c153c9c7d73 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:28.367591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93610c98-1780-4243-b33b-1c153c9c7d73 service glance] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:28.384772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1195/2391] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:33:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:33:29.450894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d4abb-c999-4316-b435-bfed3baa717a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:29.451891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d4abb-c999-4316-b435-bfed3baa717a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:29.451891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d4abb-c999-4316-b435-bfed3baa717a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:29.484655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ae8d4abb-c999-4316-b435-bfed3baa717a None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:29.486099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ae8d4abb-c999-4316-b435-bfed3baa717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:29.587733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1197/2392] 10.222.0.64 () {58 vars in 967 bytes} [Fri Sep 23 01:33:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:29.608988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93610c98-1780-4243-b33b-1c153c9c7d73 service glance] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:29.609750 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-93610c98-1780-4243-b33b-1c153c9c7d73 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:29.679734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:29.740845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93610c98-1780-4243-b33b-1c153c9c7d73 service glance] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:29.751291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053eb91-ce49-4ebf-8140-a7295ecdd631 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:29.754173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053eb91-ce49-4ebf-8140-a7295ecdd631 service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:29.754173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053eb91-ce49-4ebf-8140-a7295ecdd631 service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:29.754173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053eb91-ce49-4ebf-8140-a7295ecdd631 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:29.766392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053eb91-ce49-4ebf-8140-a7295ecdd631 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:29.768744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053eb91-ce49-4ebf-8140-a7295ecdd631 service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:29.783423 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1196/2393] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:33:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 01:33:36.072593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d4abb-c999-4316-b435-bfed3baa717a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:36.074137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ae8d4abb-c999-4316-b435-bfed3baa717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:36.131253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:36.174316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d4abb-c999-4316-b435-bfed3baa717a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:36.181690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:36.182367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:36.182658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:36.182942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] PATH_INFO: `/v3/users/1658e3b405ea41939ff42b6bb36d87f7` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:36.184880 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1658e3b405ea41939ff42b6bb36d87f7)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:36.186975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:36.516216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1658e3b405ea41939ff42b6bb36d87f7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:36.632220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1658e3b405ea41939ff42b6bb36d87f7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:36.634011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1658e3b405ea41939ff42b6bb36d87f7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:36.661825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1658e3b405ea41939ff42b6bb36d87f7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:36.663224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-cfca4f13-2d50-42ba-a3bf-4b4c4d56f2c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1658e3b405ea41939ff42b6bb36d87f7'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:36.699133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1198/2394] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:33:36 2022] DELETE /identity/v3/users/1658e3b405ea41939ff42b6bb36d87f7 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:33:37.154986 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053eb91-ce49-4ebf-8140-a7295ecdd631 service swift] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:37.173264 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053eb91-ce49-4ebf-8140-a7295ecdd631 service swift] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:37.295116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:37.298799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:37.298799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:37.298799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] PATH_INFO: `/v3/projects/3816d889342e400383f7e5e6f4ebbe04` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:37.298799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3816d889342e400383f7e5e6f4ebbe04)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:37.301245 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:37.712635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3816d889342e400383f7e5e6f4ebbe04', 'request_id': 'req-63dc65f7-58fd-4aeb-a220-d1699b07fa47'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:37.721939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3816d889342e400383f7e5e6f4ebbe04', 'request_id': 'req-63dc65f7-58fd-4aeb-a220-d1699b07fa47'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:37.843583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3816d889342e400383f7e5e6f4ebbe04', 'request_id': 'req-63dc65f7-58fd-4aeb-a220-d1699b07fa47'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:37.871818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] The token cache is being invalidate because project 3816d889342e400383f7e5e6f4ebbe04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:33:37.872045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:37.873201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-63dc65f7-58fd-4aeb-a220-d1699b07fa47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:37.875880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1197/2395] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:33:37 2022] DELETE /identity/v3/projects/3816d889342e400383f7e5e6f4ebbe04 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:33:37.882639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531330c-b66f-4569-b376-0f66b45a0c15 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:37.883618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531330c-b66f-4569-b376-0f66b45a0c15 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:37.883618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531330c-b66f-4569-b376-0f66b45a0c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:37.923028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b531330c-b66f-4569-b376-0f66b45a0c15 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:37.924855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b531330c-b66f-4569-b376-0f66b45a0c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:38.054433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1199/2396] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:33:38.060476 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bcaad-a43c-4973-b4f1-f5b1603d9e1e None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:38.071947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bcaad-a43c-4973-b4f1-f5b1603d9e1e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:38.072442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04bcaad-a43c-4973-b4f1-f5b1603d9e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:38.099690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c04bcaad-a43c-4973-b4f1-f5b1603d9e1e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:38.101111 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c04bcaad-a43c-4973-b4f1-f5b1603d9e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:38.109874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1198/2397] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:38.115409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531330c-b66f-4569-b376-0f66b45a0c15 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:38.126174 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531330c-b66f-4569-b376-0f66b45a0c15 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:38.135626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ffc8e-8725-4bce-a0c4-ac9bcebc3591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:38.137803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ffc8e-8725-4bce-a0c4-ac9bcebc3591 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:38.139934 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ffc8e-8725-4bce-a0c4-ac9bcebc3591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:38.140271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ffc8e-8725-4bce-a0c4-ac9bcebc3591 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:38.140982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ffc8e-8725-4bce-a0c4-ac9bcebc3591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:33:38.141299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ffc8e-8725-4bce-a0c4-ac9bcebc3591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:38.143011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ffc8e-8725-4bce-a0c4-ac9bcebc3591 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:38.158200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1200/2398] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:33:38.163528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bcaad-a43c-4973-b4f1-f5b1603d9e1e None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:38.175144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04bcaad-a43c-4973-b4f1-f5b1603d9e1e None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:38.181446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6de17-7149-43cc-8c96-021424033d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:38.182033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6de17-7149-43cc-8c96-021424033d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:38.182312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6de17-7149-43cc-8c96-021424033d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:38.182584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6de17-7149-43cc-8c96-021424033d21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:38.183266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6de17-7149-43cc-8c96-021424033d21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:38.185301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6de17-7149-43cc-8c96-021424033d21 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:38.461300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1199/2399] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:33:38 2022] POST /identity/v3/projects => generated 373 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:33:38.469429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ffc8e-8725-4bce-a0c4-ac9bcebc3591 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:38.491060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ffc8e-8725-4bce-a0c4-ac9bcebc3591 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:38.592070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:38.630312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77ace5-903a-4de2-9448-beeb02899701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:38.631574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77ace5-903a-4de2-9448-beeb02899701 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:38.632053 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77ace5-903a-4de2-9448-beeb02899701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:38.632428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77ace5-903a-4de2-9448-beeb02899701 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:38.633498 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77ace5-903a-4de2-9448-beeb02899701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:38.636413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77ace5-903a-4de2-9448-beeb02899701 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:39.330126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1201/2400] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:33:38 2022] POST /identity/v3/users => generated 287 bytes in 862 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:33:39.340638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6de17-7149-43cc-8c96-021424033d21 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:39.345933 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6de17-7149-43cc-8c96-021424033d21 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:39.351212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f44774-3eab-40fb-8894-f8c1484274d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:39.352038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f44774-3eab-40fb-8894-f8c1484274d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:39.352427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f44774-3eab-40fb-8894-f8c1484274d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:39.352806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f44774-3eab-40fb-8894-f8c1484274d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:39.353610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f44774-3eab-40fb-8894-f8c1484274d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:39.356129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f44774-3eab-40fb-8894-f8c1484274d5 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:39.367638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1200/2401] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:33:39.372667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77ace5-903a-4de2-9448-beeb02899701 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:39.384785 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77ace5-903a-4de2-9448-beeb02899701 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:39.393286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd7c79-df75-4afa-8235-4167df07a186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:39.393286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd7c79-df75-4afa-8235-4167df07a186 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:39.393286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd7c79-df75-4afa-8235-4167df07a186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:39.393286 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd7c79-df75-4afa-8235-4167df07a186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:39.393874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd7c79-df75-4afa-8235-4167df07a186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:39.393874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd7c79-df75-4afa-8235-4167df07a186 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:39.410177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1202/2402] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:33:39.415708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f44774-3eab-40fb-8894-f8c1484274d5 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:39.428140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f44774-3eab-40fb-8894-f8c1484274d5 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:39.435455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef521383-2a6b-496b-bf93-a5c0aa458c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:39.437407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef521383-2a6b-496b-bf93-a5c0aa458c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:39.437407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef521383-2a6b-496b-bf93-a5c0aa458c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:39.437407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef521383-2a6b-496b-bf93-a5c0aa458c35 admin admin] PATH_INFO: `/v3/projects/5cf42a933ccb4b3dac98f9ca17375b8a/users/78a942cce25d4dcda866d867a52346c9/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:39.457727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef521383-2a6b-496b-bf93-a5c0aa458c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf42a933ccb4b3dac98f9ca17375b8a, user_id=78a942cce25d4dcda866d867a52346c9, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:39.462434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef521383-2a6b-496b-bf93-a5c0aa458c35 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:39.467284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:39.563017 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1201/2403] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:33:39 2022] PUT /identity/v3/projects/5cf42a933ccb4b3dac98f9ca17375b8a/users/78a942cce25d4dcda866d867a52346c9/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:33:39.574177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae0d82-7480-4466-af44-7884464ec3c6 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:39.574711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae0d82-7480-4466-af44-7884464ec3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:39.574711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae0d82-7480-4466-af44-7884464ec3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:39.613976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e4ae0d82-7480-4466-af44-7884464ec3c6 None None] MFA Rules not processed for user `78a942cce25d4dcda866d867a52346c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:39.615765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e4ae0d82-7480-4466-af44-7884464ec3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:39.731586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1203/2404] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:39.752964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef521383-2a6b-496b-bf93-a5c0aa458c35 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:39.761783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ef521383-2a6b-496b-bf93-a5c0aa458c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:39.881012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef521383-2a6b-496b-bf93-a5c0aa458c35 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:39.885416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd7a336-dcd3-4b72-bcc3-ef0cc0edc147 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:39.886063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd7a336-dcd3-4b72-bcc3-ef0cc0edc147 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:39.886343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd7a336-dcd3-4b72-bcc3-ef0cc0edc147 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:39.886666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd7a336-dcd3-4b72-bcc3-ef0cc0edc147 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:39.896951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd7a336-dcd3-4b72-bcc3-ef0cc0edc147 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:39.898427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd7a336-dcd3-4b72-bcc3-ef0cc0edc147 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:39.910012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1202/2405] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:33:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:33:39.937716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae0d82-7480-4466-af44-7884464ec3c6 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:40.015639 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae0d82-7480-4466-af44-7884464ec3c6 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:40.031436 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:40.032178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:40.032462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:40.032773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] PATH_INFO: `/v3/users/78a942cce25d4dcda866d867a52346c9` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:40.034123 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a942cce25d4dcda866d867a52346c9)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:40.036217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:40.338047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a942cce25d4dcda866d867a52346c9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:40.346532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:40.482481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a942cce25d4dcda866d867a52346c9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:40.483287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a942cce25d4dcda866d867a52346c9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:40.500145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a942cce25d4dcda866d867a52346c9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:40.501156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a942cce25d4dcda866d867a52346c9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:40.515978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1204/2406] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:33:39 2022] DELETE /identity/v3/users/78a942cce25d4dcda866d867a52346c9 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:33:40.524526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c446f-914b-4f3b-8d7a-c96f6b45e417 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:40.526552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c446f-914b-4f3b-8d7a-c96f6b45e417 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:40.528182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c446f-914b-4f3b-8d7a-c96f6b45e417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:40.564830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f11c446f-914b-4f3b-8d7a-c96f6b45e417 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:40.566349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f11c446f-914b-4f3b-8d7a-c96f6b45e417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:40.658391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1203/2407] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:40.669558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:40.671334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:40.758113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04867190-8fcf-48a2-8a1a-c0eb1b773e26 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:40.769143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0c91a-b2b4-451c-a5b0-44224f819853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:40.770077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0c91a-b2b4-451c-a5b0-44224f819853 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:40.770556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0c91a-b2b4-451c-a5b0-44224f819853 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:40.771650 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0c91a-b2b4-451c-a5b0-44224f819853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:40.785740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c0c91a-b2b4-451c-a5b0-44224f819853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:40.787943 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c0c91a-b2b4-451c-a5b0-44224f819853 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:40.804331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1205/2408] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:33:41.547085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c446f-914b-4f3b-8d7a-c96f6b45e417 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:41.549300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f11c446f-914b-4f3b-8d7a-c96f6b45e417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:41.584707 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c446f-914b-4f3b-8d7a-c96f6b45e417 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:41.599039 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:41.600668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:41.601432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:41.603211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] PATH_INFO: `/v3/projects/5cf42a933ccb4b3dac98f9ca17375b8a` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:41.606438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cf42a933ccb4b3dac98f9ca17375b8a)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:41.611001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:41.873617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cf42a933ccb4b3dac98f9ca17375b8a', 'request_id': 'req-28edb4ac-cac6-4c9c-8726-7141b517428e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:41.882265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cf42a933ccb4b3dac98f9ca17375b8a', 'request_id': 'req-28edb4ac-cac6-4c9c-8726-7141b517428e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:41.958021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cf42a933ccb4b3dac98f9ca17375b8a', 'request_id': 'req-28edb4ac-cac6-4c9c-8726-7141b517428e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:41.975267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] The token cache is being invalidate because project 5cf42a933ccb4b3dac98f9ca17375b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:33:41.975808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:41.976943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-28edb4ac-cac6-4c9c-8726-7141b517428e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:41.979509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1204/2409] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:33:41 2022] DELETE /identity/v3/projects/5cf42a933ccb4b3dac98f9ca17375b8a => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:33:41.986888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac4a32-1b13-466f-a689-be33fbdcd199 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:41.987747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac4a32-1b13-466f-a689-be33fbdcd199 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:41.988391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac4a32-1b13-466f-a689-be33fbdcd199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:42.017156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-44ac4a32-1b13-466f-a689-be33fbdcd199 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:42.018571 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-44ac4a32-1b13-466f-a689-be33fbdcd199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:42.106108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1206/2410] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:42.114534 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502064b-9931-4490-9182-8d916b0a3e1a None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:42.114796 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502064b-9931-4490-9182-8d916b0a3e1a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:42.115136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1502064b-9931-4490-9182-8d916b0a3e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:42.159645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-1502064b-9931-4490-9182-8d916b0a3e1a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:42.161353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1502064b-9931-4490-9182-8d916b0a3e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:42.174741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1205/2411] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:42.176344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac4a32-1b13-466f-a689-be33fbdcd199 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:42.190845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac4a32-1b13-466f-a689-be33fbdcd199 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:42.198840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ee437-63b6-488b-8679-4dd94b294834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:42.200613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ee437-63b6-488b-8679-4dd94b294834 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:42.201038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ee437-63b6-488b-8679-4dd94b294834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:42.202361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56ee437-63b6-488b-8679-4dd94b294834 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:42.203635 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ee437-63b6-488b-8679-4dd94b294834 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:33:42.204020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ee437-63b6-488b-8679-4dd94b294834 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:42.208773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56ee437-63b6-488b-8679-4dd94b294834 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:42.249817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1207/2412] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:33:42.257180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502064b-9931-4490-9182-8d916b0a3e1a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:42.278068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1502064b-9931-4490-9182-8d916b0a3e1a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:42.286591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4128-9b3a-4857-aa74-a96d82bef7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:42.287417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4128-9b3a-4857-aa74-a96d82bef7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:42.291711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4128-9b3a-4857-aa74-a96d82bef7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:42.291711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e4128-9b3a-4857-aa74-a96d82bef7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:42.291711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e4128-9b3a-4857-aa74-a96d82bef7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:42.291711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e4128-9b3a-4857-aa74-a96d82bef7b7 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:42.330854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:42.617068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1206/2413] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:33:42 2022] POST /identity/v3/projects => generated 391 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:33:42.623440 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ee437-63b6-488b-8679-4dd94b294834 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:42.635629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56ee437-63b6-488b-8679-4dd94b294834 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:42.732499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e0c24-eb6a-4e3e-8e6a-439a20ebc6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:42.733298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e0c24-eb6a-4e3e-8e6a-439a20ebc6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:42.733714 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e0c24-eb6a-4e3e-8e6a-439a20ebc6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:42.734122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e0c24-eb6a-4e3e-8e6a-439a20ebc6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:42.734900 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e0c24-eb6a-4e3e-8e6a-439a20ebc6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:42.736831 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e0c24-eb6a-4e3e-8e6a-439a20ebc6c9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:42.746844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:42.924058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1208/2414] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:33:42 2022] POST /identity/v3/users => generated 296 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:33:42.929560 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4128-9b3a-4857-aa74-a96d82bef7b7 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:42.948210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e4128-9b3a-4857-aa74-a96d82bef7b7 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:42.954260 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f23e85-1281-43be-8a18-f36e8d2f7c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:42.954829 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f23e85-1281-43be-8a18-f36e8d2f7c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:42.955106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f23e85-1281-43be-8a18-f36e8d2f7c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:42.955605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f23e85-1281-43be-8a18-f36e8d2f7c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:42.956183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f23e85-1281-43be-8a18-f36e8d2f7c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:42.958118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f23e85-1281-43be-8a18-f36e8d2f7c1c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:42.971944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1207/2415] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:33:42.978171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e0c24-eb6a-4e3e-8e6a-439a20ebc6c9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:42.996312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e0c24-eb6a-4e3e-8e6a-439a20ebc6c9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:43.004109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0132ba5-dc58-4b63-923e-df6d4e736eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:43.004746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0132ba5-dc58-4b63-923e-df6d4e736eea admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:43.005065 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0132ba5-dc58-4b63-923e-df6d4e736eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:43.005358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0132ba5-dc58-4b63-923e-df6d4e736eea admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:43.006035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0132ba5-dc58-4b63-923e-df6d4e736eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:43.008512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0132ba5-dc58-4b63-923e-df6d4e736eea admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:43.047468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1209/2416] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:33:43.048048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f23e85-1281-43be-8a18-f36e8d2f7c1c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:43.057546 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f23e85-1281-43be-8a18-f36e8d2f7c1c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:43.065237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ec58-f0a0-48f0-848a-7b97d771c6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:43.066202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ec58-f0a0-48f0-848a-7b97d771c6a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:43.067236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ec58-f0a0-48f0-848a-7b97d771c6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:43.067236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ec58-f0a0-48f0-848a-7b97d771c6a4 admin admin] PATH_INFO: `/v3/projects/6fa35cc10c5244b28224b6abf8416ae7/users/ea2f6f522a154c139086e5505c6da778/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:43.107958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259ec58-f0a0-48f0-848a-7b97d771c6a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa35cc10c5244b28224b6abf8416ae7, user_id=ea2f6f522a154c139086e5505c6da778, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:43.113446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259ec58-f0a0-48f0-848a-7b97d771c6a4 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:43.285045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1208/2417] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:33:43 2022] PUT /identity/v3/projects/6fa35cc10c5244b28224b6abf8416ae7/users/ea2f6f522a154c139086e5505c6da778/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:33:43.321063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134f6cb-a998-4f5d-98ec-b653089c9ab0 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:43.328431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134f6cb-a998-4f5d-98ec-b653089c9ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:43.332617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134f6cb-a998-4f5d-98ec-b653089c9ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:43.395121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-4134f6cb-a998-4f5d-98ec-b653089c9ab0 None None] MFA Rules not processed for user `ea2f6f522a154c139086e5505c6da778`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:43.396901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4134f6cb-a998-4f5d-98ec-b653089c9ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:43.579608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1210/2418] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:43.604745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ec58-f0a0-48f0-848a-7b97d771c6a4 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:43.609126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1259ec58-f0a0-48f0-848a-7b97d771c6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:43.720290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ec58-f0a0-48f0-848a-7b97d771c6a4 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:43.725277 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abd409-8ecb-42c7-9cc8-c608b9a9af6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:43.725939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd409-8ecb-42c7-9cc8-c608b9a9af6a service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:43.726224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd409-8ecb-42c7-9cc8-c608b9a9af6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:43.726497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abd409-8ecb-42c7-9cc8-c608b9a9af6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:43.737481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abd409-8ecb-42c7-9cc8-c608b9a9af6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:43.739046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abd409-8ecb-42c7-9cc8-c608b9a9af6a service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:43.752013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1209/2419] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:33:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:33:45.404620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebcf9b-9e15-418f-8877-96c35651e485 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:45.405306 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebcf9b-9e15-418f-8877-96c35651e485 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:45.405744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebcf9b-9e15-418f-8877-96c35651e485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:45.465200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-3cebcf9b-9e15-418f-8877-96c35651e485 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:45.466729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3cebcf9b-9e15-418f-8877-96c35651e485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:45.595727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1211/2420] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:33:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:45.619359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abd409-8ecb-42c7-9cc8-c608b9a9af6a service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:45.621843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a0abd409-8ecb-42c7-9cc8-c608b9a9af6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:45.745854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abd409-8ecb-42c7-9cc8-c608b9a9af6a service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:45.751849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050e162-1dea-4588-ac3e-b7b0aa81aa86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:45.752915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050e162-1dea-4588-ac3e-b7b0aa81aa86 service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:45.753089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050e162-1dea-4588-ac3e-b7b0aa81aa86 service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:45.753517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050e162-1dea-4588-ac3e-b7b0aa81aa86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:45.770558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050e162-1dea-4588-ac3e-b7b0aa81aa86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:45.773032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a050e162-1dea-4588-ac3e-b7b0aa81aa86 service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:45.784763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1210/2421] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:33:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:33:47.412779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72ded-b42b-4cfd-926a-436d75cdcf86 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:47.413305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72ded-b42b-4cfd-926a-436d75cdcf86 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:47.413305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72ded-b42b-4cfd-926a-436d75cdcf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:47.436948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a1e72ded-b42b-4cfd-926a-436d75cdcf86 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:47.436948 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a1e72ded-b42b-4cfd-926a-436d75cdcf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:47.444485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1212/2422] 10.222.0.64 () {58 vars in 967 bytes} [Fri Sep 23 01:33:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:47.458177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050e162-1dea-4588-ac3e-b7b0aa81aa86 service swift] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:47.470596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050e162-1dea-4588-ac3e-b7b0aa81aa86 service swift] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:47.475778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c9039-f67b-4c9a-b650-36b25f01ceab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:47.476384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c9039-f67b-4c9a-b650-36b25f01ceab service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:47.476669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c9039-f67b-4c9a-b650-36b25f01ceab service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:47.476948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c9039-f67b-4c9a-b650-36b25f01ceab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:47.486487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c9039-f67b-4c9a-b650-36b25f01ceab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:47.488201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c9039-f67b-4c9a-b650-36b25f01ceab service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:47.500354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1211/2423] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:33:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:33:48.267920 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310811-21de-4c0d-909a-ae45295f0321 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:48.268214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310811-21de-4c0d-909a-ae45295f0321 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:48.268636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310811-21de-4c0d-909a-ae45295f0321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:48.291350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-13310811-21de-4c0d-909a-ae45295f0321 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:48.293240 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-13310811-21de-4c0d-909a-ae45295f0321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:48.301254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1213/2424] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:33:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:33:48.316292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c9039-f67b-4c9a-b650-36b25f01ceab service swift] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:48.337403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c9039-f67b-4c9a-b650-36b25f01ceab service swift] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:48.348211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bfda5-ab81-4db4-829e-d06bede8d11e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:48.348211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bfda5-ab81-4db4-829e-d06bede8d11e service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:48.348211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bfda5-ab81-4db4-829e-d06bede8d11e service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:48.348563 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bfda5-ab81-4db4-829e-d06bede8d11e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:48.358939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bfda5-ab81-4db4-829e-d06bede8d11e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:48.360699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bfda5-ab81-4db4-829e-d06bede8d11e service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:48.369973 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1212/2425] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:33:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:33:49.084060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f479a-77cc-4526-aa4a-0275039e6c2f None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:49.084572 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f479a-77cc-4526-aa4a-0275039e6c2f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:49.084716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f479a-77cc-4526-aa4a-0275039e6c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:49.111022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-132f479a-77cc-4526-aa4a-0275039e6c2f None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:49.112574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-132f479a-77cc-4526-aa4a-0275039e6c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:49.122452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1214/2426] 10.222.0.64 () {58 vars in 967 bytes} [Fri Sep 23 01:33:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:33:49.139336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bfda5-ab81-4db4-829e-d06bede8d11e service swift] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:49.150240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bfda5-ab81-4db4-829e-d06bede8d11e service swift] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:49.157354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062f8e4-f6ad-4c5b-807d-f4005803db5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:49.157988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f8e4-f6ad-4c5b-807d-f4005803db5d service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:49.158272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f8e4-f6ad-4c5b-807d-f4005803db5d service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:49.158548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f8e4-f6ad-4c5b-807d-f4005803db5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:49.168011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062f8e4-f6ad-4c5b-807d-f4005803db5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:49.169772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062f8e4-f6ad-4c5b-807d-f4005803db5d service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:49.189467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1213/2427] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:33:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:33:54.206407 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f479a-77cc-4526-aa4a-0275039e6c2f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:54.209538 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-132f479a-77cc-4526-aa4a-0275039e6c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:54.294729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:54.313693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f479a-77cc-4526-aa4a-0275039e6c2f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:54.320415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:54.320415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:54.320415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:54.320415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] PATH_INFO: `/v3/users/69e81cfb7aa24918b5ec8b32064e17eb` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:54.325263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] RBAC: Authorizing `identity:delete_user(user_id=69e81cfb7aa24918b5ec8b32064e17eb)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:54.325263 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:54.679729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69e81cfb7aa24918b5ec8b32064e17eb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:54.825958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69e81cfb7aa24918b5ec8b32064e17eb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:54.826666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69e81cfb7aa24918b5ec8b32064e17eb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:54.844361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69e81cfb7aa24918b5ec8b32064e17eb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:54.845378 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69e81cfb7aa24918b5ec8b32064e17eb'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:54.864550 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1215/2428] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:33:54 2022] DELETE /identity/v3/users/69e81cfb7aa24918b5ec8b32064e17eb => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:33:54.872523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14587a-49f2-4a01-987c-2334a61af421 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:54.873135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14587a-49f2-4a01-987c-2334a61af421 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:54.873135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14587a-49f2-4a01-987c-2334a61af421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:54.893303 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-0c14587a-49f2-4a01-987c-2334a61af421 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:54.894602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0c14587a-49f2-4a01-987c-2334a61af421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:54.900025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:33:54.957767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1214/2429] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:33:54.968528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:54.970482 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:55.032359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ed2bf-3903-4f11-828e-0c778699800d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:55.036734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45e76d-a4e8-428d-affe-5317decb284c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:55.037358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45e76d-a4e8-428d-affe-5317decb284c service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:55.037668 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45e76d-a4e8-428d-affe-5317decb284c service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:55.037938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45e76d-a4e8-428d-affe-5317decb284c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:55.048271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45e76d-a4e8-428d-affe-5317decb284c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:55.049741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45e76d-a4e8-428d-affe-5317decb284c service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:55.061171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1216/2430] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:33:55.361821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14587a-49f2-4a01-987c-2334a61af421 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:33:55.361821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0c14587a-49f2-4a01-987c-2334a61af421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:55.375188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14587a-49f2-4a01-987c-2334a61af421 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:33:55.382194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:33:55.383212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:55.383500 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:55.383749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] PATH_INFO: `/v3/projects/84a9bb6a15444802af12e048ea53223d` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:55.385312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a9bb6a15444802af12e048ea53223d)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:33:55.388597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:33:55.546318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a9bb6a15444802af12e048ea53223d', 'request_id': 'req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:55.559654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a9bb6a15444802af12e048ea53223d', 'request_id': 'req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:55.648513 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a9bb6a15444802af12e048ea53223d', 'request_id': 'req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:55.661508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] The token cache is being invalidate because project 84a9bb6a15444802af12e048ea53223d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:33:55.661913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:55.662798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-29d6a4d1-b42a-4482-a1f7-9ea7e913181f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:33:55.664413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1215/2431] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:33:55 2022] DELETE /identity/v3/projects/84a9bb6a15444802af12e048ea53223d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:33:58.318438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149186-7bf5-4ddf-a39a-8f846e531f4c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:33:58.320799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149186-7bf5-4ddf-a39a-8f846e531f4c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:33:58.320799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf149186-7bf5-4ddf-a39a-8f846e531f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:33:58.341445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-cf149186-7bf5-4ddf-a39a-8f846e531f4c None None] MFA Rules not processed for user `c8982a7c52314c5a82d80a03d9a833d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:33:58.342844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cf149186-7bf5-4ddf-a39a-8f846e531f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:33:58.427167 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1217/2432] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:33:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:01.068711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded020d-25b3-40b3-a873-fa2b96760cb1 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:01.069237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded020d-25b3-40b3-a873-fa2b96760cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:01.069321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded020d-25b3-40b3-a873-fa2b96760cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:01.094602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-2ded020d-25b3-40b3-a873-fa2b96760cb1 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:01.094602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2ded020d-25b3-40b3-a873-fa2b96760cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:01.216451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1216/2433] 10.222.0.64 () {58 vars in 967 bytes} [Fri Sep 23 01:34:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:01.237250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149186-7bf5-4ddf-a39a-8f846e531f4c None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:01.240829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cf149186-7bf5-4ddf-a39a-8f846e531f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:01.292299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:01.393625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf149186-7bf5-4ddf-a39a-8f846e531f4c None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:01.399743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6be3d-45d3-445b-a836-31f6915ae0e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:01.400542 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6be3d-45d3-445b-a836-31f6915ae0e0 service swift] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:01.400977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6be3d-45d3-445b-a836-31f6915ae0e0 service swift] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:01.401375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6be3d-45d3-445b-a836-31f6915ae0e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:01.413950 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6be3d-45d3-445b-a836-31f6915ae0e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:01.421637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd6be3d-45d3-445b-a836-31f6915ae0e0 service swift] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:01.438213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1218/2434] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:34:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:34:01.826327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded020d-25b3-40b3-a873-fa2b96760cb1 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:01.829138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2ded020d-25b3-40b3-a873-fa2b96760cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:01.888850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:01.915396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded020d-25b3-40b3-a873-fa2b96760cb1 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:01.921080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:01.921745 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:01.922166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:01.922424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] PATH_INFO: `/v3/users/0960a67b4dd94c8ba1c382cd9e4adfea` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:01.924217 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0960a67b4dd94c8ba1c382cd9e4adfea)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:01.926593 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:02.209516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0960a67b4dd94c8ba1c382cd9e4adfea'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:02.210668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0960a67b4dd94c8ba1c382cd9e4adfea'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:02.211896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0960a67b4dd94c8ba1c382cd9e4adfea'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:02.228702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0960a67b4dd94c8ba1c382cd9e4adfea'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:02.253654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0960a67b4dd94c8ba1c382cd9e4adfea'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:02.439644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1217/2435] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:34:01 2022] DELETE /identity/v3/users/0960a67b4dd94c8ba1c382cd9e4adfea => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:34:02.445971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23134d-81e5-4c00-ba2a-8e8cef1b479a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:02.446556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23134d-81e5-4c00-ba2a-8e8cef1b479a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:02.447295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23134d-81e5-4c00-ba2a-8e8cef1b479a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:02.477390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-1c23134d-81e5-4c00-ba2a-8e8cef1b479a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:02.478685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1c23134d-81e5-4c00-ba2a-8e8cef1b479a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:02.571124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1219/2436] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:02.597200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:02.599461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:02.694306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4631a-29c5-4510-9b40-288f87ad0e18 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:02.701122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bebdec-f7eb-432b-8bd5-5ab366087b0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:02.702669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bebdec-f7eb-432b-8bd5-5ab366087b0e service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:02.702669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bebdec-f7eb-432b-8bd5-5ab366087b0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:02.702669 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bebdec-f7eb-432b-8bd5-5ab366087b0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:02.713330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bebdec-f7eb-432b-8bd5-5ab366087b0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:02.715384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bebdec-f7eb-432b-8bd5-5ab366087b0e service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:02.727927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1218/2437] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:34:03.661121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23134d-81e5-4c00-ba2a-8e8cef1b479a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:03.663307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1c23134d-81e5-4c00-ba2a-8e8cef1b479a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:03.689970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23134d-81e5-4c00-ba2a-8e8cef1b479a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:03.698262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:03.700353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:03.700863 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:03.701352 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] PATH_INFO: `/v3/projects/9892b43bd65b4e9f9b86b14a68ed8749` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:03.704068 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9892b43bd65b4e9f9b86b14a68ed8749)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:03.708408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:03.914720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9892b43bd65b4e9f9b86b14a68ed8749', 'request_id': 'req-00f3b033-e989-4948-8bdc-4b79ad40e8d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:03.928718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9892b43bd65b4e9f9b86b14a68ed8749', 'request_id': 'req-00f3b033-e989-4948-8bdc-4b79ad40e8d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:04.031450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9892b43bd65b4e9f9b86b14a68ed8749', 'request_id': 'req-00f3b033-e989-4948-8bdc-4b79ad40e8d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:04.047922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] The token cache is being invalidate because project 9892b43bd65b4e9f9b86b14a68ed8749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:34:04.049485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:04.049485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-00f3b033-e989-4948-8bdc-4b79ad40e8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:04.056874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1220/2438] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:34:03 2022] DELETE /identity/v3/projects/9892b43bd65b4e9f9b86b14a68ed8749 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:34:04.062162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2e36-f744-4f7e-a5b1-01584aa5dbcf None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:04.062746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2e36-f744-4f7e-a5b1-01584aa5dbcf None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:04.062838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac2e36-f744-4f7e-a5b1-01584aa5dbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:04.087215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-32ac2e36-f744-4f7e-a5b1-01584aa5dbcf None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:04.088555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-32ac2e36-f744-4f7e-a5b1-01584aa5dbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:04.185465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1219/2439] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:04.195091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cfc53-b663-42f3-813f-155dc846be2a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:04.195387 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cfc53-b663-42f3-813f-155dc846be2a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:04.195930 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73cfc53-b663-42f3-813f-155dc846be2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:04.221050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b73cfc53-b663-42f3-813f-155dc846be2a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:04.222533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b73cfc53-b663-42f3-813f-155dc846be2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:04.236988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1221/2440] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:34:04.238537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2e36-f744-4f7e-a5b1-01584aa5dbcf None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:04.247350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:04.254437 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac2e36-f744-4f7e-a5b1-01584aa5dbcf None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:04.260606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d17af9-453a-40ea-a7cc-580065ee566f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:04.261494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d17af9-453a-40ea-a7cc-580065ee566f admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:04.261780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d17af9-453a-40ea-a7cc-580065ee566f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:04.262610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d17af9-453a-40ea-a7cc-580065ee566f admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:04.263642 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d17af9-453a-40ea-a7cc-580065ee566f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:34:04.264010 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d17af9-453a-40ea-a7cc-580065ee566f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:04.266333 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d17af9-453a-40ea-a7cc-580065ee566f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:04.286326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1220/2441] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:34:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:34:04.291811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73cfc53-b663-42f3-813f-155dc846be2a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:04.302679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73cfc53-b663-42f3-813f-155dc846be2a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:04.342736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018b9e0-e599-4ca7-bf0e-e39a4a3dc5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:04.342736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018b9e0-e599-4ca7-bf0e-e39a4a3dc5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:04.342736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018b9e0-e599-4ca7-bf0e-e39a4a3dc5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:04.342736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018b9e0-e599-4ca7-bf0e-e39a4a3dc5c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:04.343317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018b9e0-e599-4ca7-bf0e-e39a4a3dc5c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:04.343317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018b9e0-e599-4ca7-bf0e-e39a4a3dc5c3 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:04.791908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1222/2442] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:34:04 2022] POST /identity/v3/projects => generated 399 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:34:04.798270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d17af9-453a-40ea-a7cc-580065ee566f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:04.810816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d17af9-453a-40ea-a7cc-580065ee566f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:04.887848 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ac86f-7153-4918-8392-8f60d6ed3cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:04.888923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ac86f-7153-4918-8392-8f60d6ed3cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:04.889428 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ac86f-7153-4918-8392-8f60d6ed3cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:04.889950 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ac86f-7153-4918-8392-8f60d6ed3cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:04.891009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ac86f-7153-4918-8392-8f60d6ed3cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:04.893087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ac86f-7153-4918-8392-8f60d6ed3cb1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:05.007251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1221/2443] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:34:04 2022] POST /identity/v3/users => generated 300 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:34:05.011725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018b9e0-e599-4ca7-bf0e-e39a4a3dc5c3 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:05.021029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:05.027392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018b9e0-e599-4ca7-bf0e-e39a4a3dc5c3 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:05.034401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6f10a-a8cb-47b3-9677-3a01e775070b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:05.035116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6f10a-a8cb-47b3-9677-3a01e775070b admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:05.035471 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6f10a-a8cb-47b3-9677-3a01e775070b admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:05.035786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6f10a-a8cb-47b3-9677-3a01e775070b admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:05.036442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6f10a-a8cb-47b3-9677-3a01e775070b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:05.038059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6f10a-a8cb-47b3-9677-3a01e775070b admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:05.057003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1223/2444] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:34:05.057122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ac86f-7153-4918-8392-8f60d6ed3cb1 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:05.068999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ac86f-7153-4918-8392-8f60d6ed3cb1 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:05.074645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c675702-3e41-4e49-84ab-a5a5c6a70d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:05.075469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675702-3e41-4e49-84ab-a5a5c6a70d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:05.075727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675702-3e41-4e49-84ab-a5a5c6a70d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:05.076187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c675702-3e41-4e49-84ab-a5a5c6a70d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:05.076776 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c675702-3e41-4e49-84ab-a5a5c6a70d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:05.078759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c675702-3e41-4e49-84ab-a5a5c6a70d62 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:05.096232 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1222/2445] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:34:05.103454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6f10a-a8cb-47b3-9677-3a01e775070b admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:05.117719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6f10a-a8cb-47b3-9677-3a01e775070b admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:05.124764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba385d-df26-4f3f-a3a7-ad9cebf91644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:05.126324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba385d-df26-4f3f-a3a7-ad9cebf91644 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:05.126901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba385d-df26-4f3f-a3a7-ad9cebf91644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:05.127524 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba385d-df26-4f3f-a3a7-ad9cebf91644 admin admin] PATH_INFO: `/v3/projects/7f2b4aaba35d4395abdeca782b9f367f/users/63377f10c7994ecda185611efb45606a/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:05.148690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba385d-df26-4f3f-a3a7-ad9cebf91644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f2b4aaba35d4395abdeca782b9f367f, user_id=63377f10c7994ecda185611efb45606a, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:05.152450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba385d-df26-4f3f-a3a7-ad9cebf91644 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:05.260737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1224/2446] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:34:05 2022] PUT /identity/v3/projects/7f2b4aaba35d4395abdeca782b9f367f/users/63377f10c7994ecda185611efb45606a/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:34:05.279310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece346a2-12b9-409e-a6fa-2fa9458806b5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:05.279771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece346a2-12b9-409e-a6fa-2fa9458806b5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:05.280298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece346a2-12b9-409e-a6fa-2fa9458806b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:05.330617 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ece346a2-12b9-409e-a6fa-2fa9458806b5 None None] MFA Rules not processed for user `63377f10c7994ecda185611efb45606a`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:05.332014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ece346a2-12b9-409e-a6fa-2fa9458806b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:05.414679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1223/2447] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:05 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:34:05.427085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba385d-df26-4f3f-a3a7-ad9cebf91644 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:05.429508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-25ba385d-df26-4f3f-a3a7-ad9cebf91644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:05.512553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba385d-df26-4f3f-a3a7-ad9cebf91644 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:05.516887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b4cb6-e4a6-4788-b217-9d7286ffa8b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:05.517867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b4cb6-e4a6-4788-b217-9d7286ffa8b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:05.517867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b4cb6-e4a6-4788-b217-9d7286ffa8b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:05.518146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b4cb6-e4a6-4788-b217-9d7286ffa8b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:05.529295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b4cb6-e4a6-4788-b217-9d7286ffa8b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:05.531135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b4cb6-e4a6-4788-b217-9d7286ffa8b4 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:05.542903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1225/2448] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:34:05 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:34:15.411565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:15.481954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece346a2-12b9-409e-a6fa-2fa9458806b5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:15.536868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:15.559858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece346a2-12b9-409e-a6fa-2fa9458806b5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:15.564970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:15.565635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:15.565919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:15.566197 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] PATH_INFO: `/v3/users/63377f10c7994ecda185611efb45606a` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:15.567386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=63377f10c7994ecda185611efb45606a)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:15.569522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:15.624169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b4cb6-e4a6-4788-b217-9d7286ffa8b4 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:15.626272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-238b4cb6-e4a6-4788-b217-9d7286ffa8b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:15.646708 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b4cb6-e4a6-4788-b217-9d7286ffa8b4 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:15.653294 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:15.658127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:15.658127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:15.658127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] PATH_INFO: `/v3/users/ea2f6f522a154c139086e5505c6da778` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:15.658127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2f6f522a154c139086e5505c6da778)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:15.658127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:15.837105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63377f10c7994ecda185611efb45606a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:15.837918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63377f10c7994ecda185611efb45606a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:15.838646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63377f10c7994ecda185611efb45606a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:15.850893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63377f10c7994ecda185611efb45606a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:15.868310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2492d4d7-5f5d-4dfd-92a8-069f88180d3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63377f10c7994ecda185611efb45606a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:15.995570 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1224/2449] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:34:15 2022] DELETE /identity/v3/users/63377f10c7994ecda185611efb45606a => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:34:16.000592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2f6f522a154c139086e5505c6da778'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:16.003231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a564c1-57dd-4f66-a38f-801789fe928b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:16.004022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a564c1-57dd-4f66-a38f-801789fe928b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:16.004562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a564c1-57dd-4f66-a38f-801789fe928b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:16.042112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f8a564c1-57dd-4f66-a38f-801789fe928b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:16.045132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f8a564c1-57dd-4f66-a38f-801789fe928b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:16.083459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2f6f522a154c139086e5505c6da778'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:16.084249 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2f6f522a154c139086e5505c6da778'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:16.109911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2f6f522a154c139086e5505c6da778'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:16.110772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-621259be-89e0-4836-86ea-9f33bb3d473f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2f6f522a154c139086e5505c6da778'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:16.138216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1226/2450] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:34:15 2022] DELETE /identity/v3/users/ea2f6f522a154c139086e5505c6da778 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:34:16.145142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ff495-af2b-49de-a7d8-49718b4685bb None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:16.149274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ff495-af2b-49de-a7d8-49718b4685bb None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:16.149274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ff495-af2b-49de-a7d8-49718b4685bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:16.157494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1225/2451] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:16.169247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a564c1-57dd-4f66-a38f-801789fe928b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:16.171594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f8a564c1-57dd-4f66-a38f-801789fe928b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:16.182664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-391ff495-af2b-49de-a7d8-49718b4685bb None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:16.185458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-391ff495-af2b-49de-a7d8-49718b4685bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:16.211908 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:16.284283 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a564c1-57dd-4f66-a38f-801789fe928b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:16.302390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1227/2452] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:16.320419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ff495-af2b-49de-a7d8-49718b4685bb None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:16.375828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87612af-a25c-4bf9-8ecf-3842ac4e6ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:16.376601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87612af-a25c-4bf9-8ecf-3842ac4e6ace service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:16.376780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87612af-a25c-4bf9-8ecf-3842ac4e6ace service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:16.377057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87612af-a25c-4bf9-8ecf-3842ac4e6ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:16.392052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87612af-a25c-4bf9-8ecf-3842ac4e6ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:16.394740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87612af-a25c-4bf9-8ecf-3842ac4e6ace service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:16.404559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ff495-af2b-49de-a7d8-49718b4685bb None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:16.407944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1226/2453] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:34:16.411849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb761b-cb34-4ae5-ba7e-8ad9bed7d6ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:16.412903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb761b-cb34-4ae5-ba7e-8ad9bed7d6ce service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:16.413383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb761b-cb34-4ae5-ba7e-8ad9bed7d6ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:16.413787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb761b-cb34-4ae5-ba7e-8ad9bed7d6ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:16.431812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb761b-cb34-4ae5-ba7e-8ad9bed7d6ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:16.434599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb761b-cb34-4ae5-ba7e-8ad9bed7d6ce service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:16.451938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1228/2454] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:34:17.363914 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87612af-a25c-4bf9-8ecf-3842ac4e6ace service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:17.366319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b87612af-a25c-4bf9-8ecf-3842ac4e6ace service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:17.383609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87612af-a25c-4bf9-8ecf-3842ac4e6ace service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:17.391717 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:17.392547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:17.393071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:17.393647 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] PATH_INFO: `/v3/projects/7f2b4aaba35d4395abdeca782b9f367f` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:17.395909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f2b4aaba35d4395abdeca782b9f367f)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:17.400109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:17.424723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb761b-cb34-4ae5-ba7e-8ad9bed7d6ce service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:17.430255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7cfb761b-cb34-4ae5-ba7e-8ad9bed7d6ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:17.472888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb761b-cb34-4ae5-ba7e-8ad9bed7d6ce service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:17.494154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:17.494154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:17.494154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:17.494154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] PATH_INFO: `/v3/projects/6fa35cc10c5244b28224b6abf8416ae7` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:17.495848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fa35cc10c5244b28224b6abf8416ae7)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:17.496308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:17.513906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:17.674925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f2b4aaba35d4395abdeca782b9f367f', 'request_id': 'req-a8a28902-37e1-4d19-a666-48b0a363c7cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:17.687450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f2b4aaba35d4395abdeca782b9f367f', 'request_id': 'req-a8a28902-37e1-4d19-a666-48b0a363c7cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:17.702752 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fa35cc10c5244b28224b6abf8416ae7', 'request_id': 'req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:17.710026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fa35cc10c5244b28224b6abf8416ae7', 'request_id': 'req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:17.791336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f2b4aaba35d4395abdeca782b9f367f', 'request_id': 'req-a8a28902-37e1-4d19-a666-48b0a363c7cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:17.807490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] The token cache is being invalidate because project 7f2b4aaba35d4395abdeca782b9f367f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:34:17.807765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:17.808405 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-a8a28902-37e1-4d19-a666-48b0a363c7cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:17.809968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1227/2455] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:34:17 2022] DELETE /identity/v3/projects/7f2b4aaba35d4395abdeca782b9f367f => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:34:17.819780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca80ee-caa2-4247-8440-b96b42ce0aa5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:17.820162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca80ee-caa2-4247-8440-b96b42ce0aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:17.820529 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca80ee-caa2-4247-8440-b96b42ce0aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:17.847877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-8cca80ee-caa2-4247-8440-b96b42ce0aa5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:17.849151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-8cca80ee-caa2-4247-8440-b96b42ce0aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:17.871505 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fa35cc10c5244b28224b6abf8416ae7', 'request_id': 'req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:17.896747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] The token cache is being invalidate because project 6fa35cc10c5244b28224b6abf8416ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:34:17.897587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:17.898721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b9ab8769-b26b-4ca4-ab78-7a2ab28451fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:34:17.900924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1229/2456] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:34:17 2022] DELETE /identity/v3/projects/6fa35cc10c5244b28224b6abf8416ae7 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:34:17.935644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1228/2457] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:34:17.949641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567112e-353f-4b7a-bffb-46927aad99e6 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:17.950110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567112e-353f-4b7a-bffb-46927aad99e6 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:17.951290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567112e-353f-4b7a-bffb-46927aad99e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:17.986937 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-c567112e-353f-4b7a-bffb-46927aad99e6 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:17.988725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c567112e-353f-4b7a-bffb-46927aad99e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:18.000458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1230/2458] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:18.008825 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca80ee-caa2-4247-8440-b96b42ce0aa5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:18.015575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:18.025606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca80ee-caa2-4247-8440-b96b42ce0aa5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:18.032355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c1a33-fb72-45c6-aab7-9157917b9c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:18.033278 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c1a33-fb72-45c6-aab7-9157917b9c38 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:18.033727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c1a33-fb72-45c6-aab7-9157917b9c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:18.034897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c1a33-fb72-45c6-aab7-9157917b9c38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:18.035349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c1a33-fb72-45c6-aab7-9157917b9c38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:34:18.035882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c1a33-fb72-45c6-aab7-9157917b9c38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:18.038568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c1a33-fb72-45c6-aab7-9157917b9c38 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:18.065055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1229/2459] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:34:18.065353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567112e-353f-4b7a-bffb-46927aad99e6 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:18.079210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567112e-353f-4b7a-bffb-46927aad99e6 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:18.089807 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504839d-a09f-4e26-9559-2c30a2501431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:18.090922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504839d-a09f-4e26-9559-2c30a2501431 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:18.091427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504839d-a09f-4e26-9559-2c30a2501431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:18.091965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504839d-a09f-4e26-9559-2c30a2501431 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:18.096259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504839d-a09f-4e26-9559-2c30a2501431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:18.096259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8504839d-a09f-4e26-9559-2c30a2501431 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:18.207261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1231/2460] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:34:18 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:34:18.212145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c1a33-fb72-45c6-aab7-9157917b9c38 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:18.224523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c1a33-fb72-45c6-aab7-9157917b9c38 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:18.298466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a1452-c737-4148-8148-27b20be6b152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:18.299182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a1452-c737-4148-8148-27b20be6b152 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:18.299508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a1452-c737-4148-8148-27b20be6b152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:18.300177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a1452-c737-4148-8148-27b20be6b152 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:18.300679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a1452-c737-4148-8148-27b20be6b152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:18.303003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a1452-c737-4148-8148-27b20be6b152 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:18.441001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1230/2461] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:34:18 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:34:18.451817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504839d-a09f-4e26-9559-2c30a2501431 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:18.473182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8504839d-a09f-4e26-9559-2c30a2501431 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:18.481585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de015f-0a6d-4b01-8dc2-34b3914de1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:18.482568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de015f-0a6d-4b01-8dc2-34b3914de1c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:18.483004 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de015f-0a6d-4b01-8dc2-34b3914de1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:18.487638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de015f-0a6d-4b01-8dc2-34b3914de1c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:18.487638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de015f-0a6d-4b01-8dc2-34b3914de1c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:18.487638 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de015f-0a6d-4b01-8dc2-34b3914de1c7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:18.509587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1232/2462] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:34:18.513871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a1452-c737-4148-8148-27b20be6b152 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:18.525990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a1452-c737-4148-8148-27b20be6b152 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:18.531881 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a50122-2762-40bb-88e9-8f6392312293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:18.532511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a50122-2762-40bb-88e9-8f6392312293 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:18.532788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a50122-2762-40bb-88e9-8f6392312293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:18.533116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a50122-2762-40bb-88e9-8f6392312293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:18.533818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a50122-2762-40bb-88e9-8f6392312293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:18.535484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a50122-2762-40bb-88e9-8f6392312293 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:18.550446 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1231/2463] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:34:18.555716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de015f-0a6d-4b01-8dc2-34b3914de1c7 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:18.581393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de015f-0a6d-4b01-8dc2-34b3914de1c7 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:18.587317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac7c8a-c839-41cf-91c8-048972a7f9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:18.588231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac7c8a-c839-41cf-91c8-048972a7f9d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:18.588605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac7c8a-c839-41cf-91c8-048972a7f9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:18.589405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac7c8a-c839-41cf-91c8-048972a7f9d6 admin admin] PATH_INFO: `/v3/projects/600dd39d7a7944a49330866c0e26cb66/users/8089cf57e0014489a12e659278c2931e/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:18.610846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac7c8a-c839-41cf-91c8-048972a7f9d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600dd39d7a7944a49330866c0e26cb66, user_id=8089cf57e0014489a12e659278c2931e, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:18.613558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac7c8a-c839-41cf-91c8-048972a7f9d6 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:18.713586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1233/2464] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:34:18 2022] PUT /identity/v3/projects/600dd39d7a7944a49330866c0e26cb66/users/8089cf57e0014489a12e659278c2931e/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:34:18.722029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cec7e-ec9c-4d50-b7a4-2ebd98b910a0 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:18.722412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cec7e-ec9c-4d50-b7a4-2ebd98b910a0 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:18.722675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cec7e-ec9c-4d50-b7a4-2ebd98b910a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:18.747148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-9c0cec7e-ec9c-4d50-b7a4-2ebd98b910a0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:18.748497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-9c0cec7e-ec9c-4d50-b7a4-2ebd98b910a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:18.871189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1232/2465] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:18.884748 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac7c8a-c839-41cf-91c8-048972a7f9d6 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:18.888523 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-efac7c8a-c839-41cf-91c8-048972a7f9d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:18.992058 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac7c8a-c839-41cf-91c8-048972a7f9d6 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:18.997430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64a11-1e64-46e8-baf2-087dd4606f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:18.997430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64a11-1e64-46e8-baf2-087dd4606f4d service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:18.997430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64a11-1e64-46e8-baf2-087dd4606f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:18.997986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64a11-1e64-46e8-baf2-087dd4606f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:19.009696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64a11-1e64-46e8-baf2-087dd4606f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:19.011962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64a11-1e64-46e8-baf2-087dd4606f4d service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:19.024405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1234/2466] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:34:26.650849 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66e54-35f7-45d1-bff6-bd6c403c7c27 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:26.652196 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66e54-35f7-45d1-bff6-bd6c403c7c27 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:26.652196 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf66e54-35f7-45d1-bff6-bd6c403c7c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:26.687053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-fbf66e54-35f7-45d1-bff6-bd6c403c7c27 None None] MFA Rules not processed for user `8089cf57e0014489a12e659278c2931e`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:26.688316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-fbf66e54-35f7-45d1-bff6-bd6c403c7c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:26.762948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:26.772168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1233/2467] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:26.780416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64a11-1e64-46e8-baf2-087dd4606f4d service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:26.795457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64a11-1e64-46e8-baf2-087dd4606f4d service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:26.808645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a32d5-7335-49ec-828b-04b2fa588195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:26.808645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a32d5-7335-49ec-828b-04b2fa588195 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:26.809928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a32d5-7335-49ec-828b-04b2fa588195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:26.809928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a32d5-7335-49ec-828b-04b2fa588195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:26.809928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a32d5-7335-49ec-828b-04b2fa588195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:26.811924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a32d5-7335-49ec-828b-04b2fa588195 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:26.916772 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1235/2468] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:34:26 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:34:26.922763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf66e54-35f7-45d1-bff6-bd6c403c7c27 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:26.937482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf66e54-35f7-45d1-bff6-bd6c403c7c27 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:27.030297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a823d-ca49-4337-bdfd-5242b74c3628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:27.031332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a823d-ca49-4337-bdfd-5242b74c3628 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:27.031828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a823d-ca49-4337-bdfd-5242b74c3628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:27.032140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a823d-ca49-4337-bdfd-5242b74c3628 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:27.032945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a823d-ca49-4337-bdfd-5242b74c3628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:27.035205 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a823d-ca49-4337-bdfd-5242b74c3628 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:27.141456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1234/2469] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:34:26 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:34:27.146602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a32d5-7335-49ec-828b-04b2fa588195 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:27.159854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a32d5-7335-49ec-828b-04b2fa588195 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:27.168348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1333f-2228-4ecd-b35b-22289cd5887f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:27.169382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1333f-2228-4ecd-b35b-22289cd5887f admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:27.169877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1333f-2228-4ecd-b35b-22289cd5887f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:27.170353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1333f-2228-4ecd-b35b-22289cd5887f admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:27.171599 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1333f-2228-4ecd-b35b-22289cd5887f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:27.174656 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1333f-2228-4ecd-b35b-22289cd5887f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:27.190385 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1236/2470] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:34:27.197052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a823d-ca49-4337-bdfd-5242b74c3628 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:27.216678 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a823d-ca49-4337-bdfd-5242b74c3628 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:27.223130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2be6d-87a8-4977-a3fc-e9603facaa4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:27.223877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2be6d-87a8-4977-a3fc-e9603facaa4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:27.224164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2be6d-87a8-4977-a3fc-e9603facaa4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:27.224477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2be6d-87a8-4977-a3fc-e9603facaa4c admin admin] PATH_INFO: `/v3/projects/81dd2c4296a5457894420a8114b1ab5d/users/c92a1ec9f82a4fd3886b67bf743529cd/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:27.248289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2be6d-87a8-4977-a3fc-e9603facaa4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=81dd2c4296a5457894420a8114b1ab5d, user_id=c92a1ec9f82a4fd3886b67bf743529cd, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:27.255089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2be6d-87a8-4977-a3fc-e9603facaa4c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:27.372223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1235/2471] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:34:27 2022] PUT /identity/v3/projects/81dd2c4296a5457894420a8114b1ab5d/users/c92a1ec9f82a4fd3886b67bf743529cd/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:34:27.377313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1333f-2228-4ecd-b35b-22289cd5887f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:27.386925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:27.396072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1333f-2228-4ecd-b35b-22289cd5887f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:27.462964 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cab829-6efe-4683-9b6c-b8adb13f82ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:27.463706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cab829-6efe-4683-9b6c-b8adb13f82ca admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:27.464779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cab829-6efe-4683-9b6c-b8adb13f82ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:27.464984 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cab829-6efe-4683-9b6c-b8adb13f82ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:27.465644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cab829-6efe-4683-9b6c-b8adb13f82ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:27.467282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cab829-6efe-4683-9b6c-b8adb13f82ca admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:27.480436 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1237/2472] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:34:27.484793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2be6d-87a8-4977-a3fc-e9603facaa4c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:27.497548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2be6d-87a8-4977-a3fc-e9603facaa4c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:27.503923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a1d03-6e19-4286-af54-0a4288992f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:27.504397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a1d03-6e19-4286-af54-0a4288992f01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:27.504679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a1d03-6e19-4286-af54-0a4288992f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:27.504977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a1d03-6e19-4286-af54-0a4288992f01 admin admin] PATH_INFO: `/v3/projects/81dd2c4296a5457894420a8114b1ab5d/users/c92a1ec9f82a4fd3886b67bf743529cd/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:27.507435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a1d03-6e19-4286-af54-0a4288992f01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81dd2c4296a5457894420a8114b1ab5d, user_id=c92a1ec9f82a4fd3886b67bf743529cd, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:27.512220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a1d03-6e19-4286-af54-0a4288992f01 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:27.534084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1236/2473] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:34:27 2022] PUT /identity/v3/projects/81dd2c4296a5457894420a8114b1ab5d/users/c92a1ec9f82a4fd3886b67bf743529cd/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:34:34.825533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954ce61-2616-4f1d-84f2-c0db98e7a374 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:34.825995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954ce61-2616-4f1d-84f2-c0db98e7a374 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:34.826373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954ce61-2616-4f1d-84f2-c0db98e7a374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:34.862251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-6954ce61-2616-4f1d-84f2-c0db98e7a374 None None] MFA Rules not processed for user `c92a1ec9f82a4fd3886b67bf743529cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:34.872578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6954ce61-2616-4f1d-84f2-c0db98e7a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:34.946939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1238/2474] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:34.961871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a1d03-6e19-4286-af54-0a4288992f01 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:34.964495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ed9a1d03-6e19-4286-af54-0a4288992f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:35.051860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a1d03-6e19-4286-af54-0a4288992f01 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:35.061342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef38f3b-e294-4b00-84b8-047260f3e2f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:35.061985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef38f3b-e294-4b00-84b8-047260f3e2f6 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:35.062272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef38f3b-e294-4b00-84b8-047260f3e2f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:35.062586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef38f3b-e294-4b00-84b8-047260f3e2f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:35.119508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:35.132322 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef38f3b-e294-4b00-84b8-047260f3e2f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:35.133960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef38f3b-e294-4b00-84b8-047260f3e2f6 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:35.144049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1237/2475] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:34:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:34:39.140960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021113a-34ac-490f-b169-3bc89880f85a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:39.140960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021113a-34ac-490f-b169-3bc89880f85a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:39.140960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021113a-34ac-490f-b169-3bc89880f85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:39.166031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-2021113a-34ac-490f-b169-3bc89880f85a None None] MFA Rules not processed for user `8089cf57e0014489a12e659278c2931e`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:39.167418 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2021113a-34ac-490f-b169-3bc89880f85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:39.175525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1239/2476] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:34:42.112162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef38f3b-e294-4b00-84b8-047260f3e2f6 service nova] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:42.115444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6ef38f3b-e294-4b00-84b8-047260f3e2f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:42.186369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef38f3b-e294-4b00-84b8-047260f3e2f6 service nova] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:42.190418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6e2b6-2644-41f2-af8c-0d5a50d7d92e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:42.191077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6e2b6-2644-41f2-af8c-0d5a50d7d92e service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:42.191350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6e2b6-2644-41f2-af8c-0d5a50d7d92e service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:42.191661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6e2b6-2644-41f2-af8c-0d5a50d7d92e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:42.193578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-19b6e2b6-2644-41f2-af8c-0d5a50d7d92e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:42.208099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6e2b6-2644-41f2-af8c-0d5a50d7d92e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:42.209962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b6e2b6-2644-41f2-af8c-0d5a50d7d92e service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:42.223346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1238/2477] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:34:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:34:43.449111 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021113a-34ac-490f-b169-3bc89880f85a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:34:43.468055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021113a-34ac-490f-b169-3bc89880f85a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:34:43.472760 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31195be7-163e-4707-bbde-ece39a2c9673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:34:43.473472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31195be7-163e-4707-bbde-ece39a2c9673 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:43.473808 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31195be7-163e-4707-bbde-ece39a2c9673 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:43.474121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31195be7-163e-4707-bbde-ece39a2c9673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:43.476253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-31195be7-163e-4707-bbde-ece39a2c9673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:43.490909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31195be7-163e-4707-bbde-ece39a2c9673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:34:43.492406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31195be7-163e-4707-bbde-ece39a2c9673 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:34:43.504878 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1240/2478] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:34:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:34:52.218313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=32 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:53.499308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=70 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:34:55.916007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141a0bd-f598-481b-8285-1345534a19b8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:55.916877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141a0bd-f598-481b-8285-1345534a19b8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:55.916877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141a0bd-f598-481b-8285-1345534a19b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:55.938631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-7141a0bd-f598-481b-8285-1345534a19b8 None None] MFA Rules not processed for user `da9978cab3fb4bdda39a2e4718ee2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:55.939916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7141a0bd-f598-481b-8285-1345534a19b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:56.003529 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1239/2479] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:34:58.342702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc4859-c318-4c66-9880-e3fed46a600b None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:34:58.343253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc4859-c318-4c66-9880-e3fed46a600b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:34:58.343766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc4859-c318-4c66-9880-e3fed46a600b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:34:58.376493 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-d8dc4859-c318-4c66-9880-e3fed46a600b None None] MFA Rules not processed for user `62f175cb5d1e4ac8a0ee95cf7718837b`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:34:58.377903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d8dc4859-c318-4c66-9880-e3fed46a600b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:34:58.439877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1241/2480] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:34:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:35:00.238719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141a0bd-f598-481b-8285-1345534a19b8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:00.241029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7141a0bd-f598-481b-8285-1345534a19b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:00.331867 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141a0bd-f598-481b-8285-1345534a19b8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:00.338580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a039c-1794-466f-b577-d312f6432414 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:00.339271 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a039c-1794-466f-b577-d312f6432414 service placement] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:00.339581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a039c-1794-466f-b577-d312f6432414 service placement] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:00.339894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a039c-1794-466f-b577-d312f6432414 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:00.343095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-116a039c-1794-466f-b577-d312f6432414 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:00.360686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a039c-1794-466f-b577-d312f6432414 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:00.360686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a039c-1794-466f-b577-d312f6432414 service placement] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:00.370413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1240/2481] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:35:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:35:00.787168 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc4859-c318-4c66-9880-e3fed46a600b None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:00.798123 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc4859-c318-4c66-9880-e3fed46a600b None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:00.801885 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2d0a-f64d-4c37-9af7-dfdb5f8e79ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:00.802485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2d0a-f64d-4c37-9af7-dfdb5f8e79ab service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:00.802766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2d0a-f64d-4c37-9af7-dfdb5f8e79ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:00.803041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2d0a-f64d-4c37-9af7-dfdb5f8e79ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:00.804894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-965d2d0a-f64d-4c37-9af7-dfdb5f8e79ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:00.817022 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d2d0a-f64d-4c37-9af7-dfdb5f8e79ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:00.819915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d2d0a-f64d-4c37-9af7-dfdb5f8e79ab service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:00.833301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1242/2482] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:35:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:35:01.316684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a039c-1794-466f-b577-d312f6432414 service placement] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:01.318176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-116a039c-1794-466f-b577-d312f6432414 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:01.334235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a039c-1794-466f-b577-d312f6432414 service placement] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:01.340657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cac90-f32e-4e5e-82b2-f7a772fc8bb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:01.348510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cac90-f32e-4e5e-82b2-f7a772fc8bb0 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:01.348510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cac90-f32e-4e5e-82b2-f7a772fc8bb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:01.348510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cac90-f32e-4e5e-82b2-f7a772fc8bb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:01.348510 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b82cac90-f32e-4e5e-82b2-f7a772fc8bb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:01.356761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cac90-f32e-4e5e-82b2-f7a772fc8bb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:01.358376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cac90-f32e-4e5e-82b2-f7a772fc8bb0 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:01.368387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1241/2483] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:35:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:35:02.689219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2d0a-f64d-4c37-9af7-dfdb5f8e79ab service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:02.715393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2d0a-f64d-4c37-9af7-dfdb5f8e79ab service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:02.715393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43fa19-35bd-4d3c-b3ce-4c7996b72bc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:02.715393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43fa19-35bd-4d3c-b3ce-4c7996b72bc8 service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:02.715393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43fa19-35bd-4d3c-b3ce-4c7996b72bc8 service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:02.715949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43fa19-35bd-4d3c-b3ce-4c7996b72bc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:02.715949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cd43fa19-35bd-4d3c-b3ce-4c7996b72bc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:02.724213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43fa19-35bd-4d3c-b3ce-4c7996b72bc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:02.725962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43fa19-35bd-4d3c-b3ce-4c7996b72bc8 service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:02.736756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1243/2484] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:35:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:35:02.853093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cac90-f32e-4e5e-82b2-f7a772fc8bb0 service nova] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:02.853093 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b82cac90-f32e-4e5e-82b2-f7a772fc8bb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:02.937327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cac90-f32e-4e5e-82b2-f7a772fc8bb0 service nova] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:02.941821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b465a68-1069-4fb8-b3f7-8f88a3522889 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:02.942721 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465a68-1069-4fb8-b3f7-8f88a3522889 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:02.950872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465a68-1069-4fb8-b3f7-8f88a3522889 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:02.950872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465a68-1069-4fb8-b3f7-8f88a3522889 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:02.950872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-7b465a68-1069-4fb8-b3f7-8f88a3522889 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:02.998692 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:35:03.025363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b465a68-1069-4fb8-b3f7-8f88a3522889 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:03.026821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b465a68-1069-4fb8-b3f7-8f88a3522889 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:03.039383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1242/2485] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:35:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:35:12.732220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=37 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:35:13.217343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=11 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:35:16.143792 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43fa19-35bd-4d3c-b3ce-4c7996b72bc8 service nova] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:16.153909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43fa19-35bd-4d3c-b3ce-4c7996b72bc8 service nova] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:16.157631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beab043-f3b4-42af-9510-3a86e1576a6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:16.158244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beab043-f3b4-42af-9510-3a86e1576a6e service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:16.158525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beab043-f3b4-42af-9510-3a86e1576a6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:16.158801 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beab043-f3b4-42af-9510-3a86e1576a6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:16.160922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6beab043-f3b4-42af-9510-3a86e1576a6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:16.171843 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beab043-f3b4-42af-9510-3a86e1576a6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:16.173402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beab043-f3b4-42af-9510-3a86e1576a6e service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:16.181790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1244/2486] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:35:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:35:19.562905 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e32f5-cc5f-4c08-831d-1e828e19dd09 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:19.563475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e32f5-cc5f-4c08-831d-1e828e19dd09 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:19.563475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e32f5-cc5f-4c08-831d-1e828e19dd09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:19.583379 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-1b1e32f5-cc5f-4c08-831d-1e828e19dd09 None None] MFA Rules not processed for user `1633ea35eda34ffe884140eaab1ae909`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:35:19.589656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1b1e32f5-cc5f-4c08-831d-1e828e19dd09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:19.591705 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1243/2487] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:35:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:35:26.178121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=6 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:35:26.382235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beab043-f3b4-42af-9510-3a86e1576a6e service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:26.383902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6beab043-f3b4-42af-9510-3a86e1576a6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:26.467580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beab043-f3b4-42af-9510-3a86e1576a6e service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:26.475478 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:26.476187 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:26.476469 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:26.476746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] PATH_INFO: `/v3/users/c8982a7c52314c5a82d80a03d9a833d6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:26.477967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8982a7c52314c5a82d80a03d9a833d6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:26.480055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:26.772774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8982a7c52314c5a82d80a03d9a833d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:35:26.843670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e32f5-cc5f-4c08-831d-1e828e19dd09 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:26.861650 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e32f5-cc5f-4c08-831d-1e828e19dd09 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:26.868805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ee0f8-f710-46b7-82a7-23faeea70307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:26.869655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ee0f8-f710-46b7-82a7-23faeea70307 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:26.870040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ee0f8-f710-46b7-82a7-23faeea70307 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:26.871112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ee0f8-f710-46b7-82a7-23faeea70307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:26.873457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-530ee0f8-f710-46b7-82a7-23faeea70307 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:26.888342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ee0f8-f710-46b7-82a7-23faeea70307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:26.890545 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ee0f8-f710-46b7-82a7-23faeea70307 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:26.903098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1244/2488] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:35:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:35:26.950920 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8982a7c52314c5a82d80a03d9a833d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:35:26.950920 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8982a7c52314c5a82d80a03d9a833d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:35:26.963497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8982a7c52314c5a82d80a03d9a833d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:35:26.965460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8982a7c52314c5a82d80a03d9a833d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:35:27.028554 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1245/2489] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:35:26 2022] DELETE /identity/v3/users/c8982a7c52314c5a82d80a03d9a833d6 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:35:27.632541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ee0f8-f710-46b7-82a7-23faeea70307 service nova] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:27.652826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ee0f8-f710-46b7-82a7-23faeea70307 service nova] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:27.782356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:27.783744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:27.784199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:27.787298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] PATH_INFO: `/v3/projects/95935852764e413c8bbd23c16ddf13a5` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:27.787298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95935852764e413c8bbd23c16ddf13a5)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:27.788766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:27.991517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95935852764e413c8bbd23c16ddf13a5', 'request_id': 'req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:35:28.011130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95935852764e413c8bbd23c16ddf13a5', 'request_id': 'req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:35:28.115382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95935852764e413c8bbd23c16ddf13a5', 'request_id': 'req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:35:28.133289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] The token cache is being invalidate because project 95935852764e413c8bbd23c16ddf13a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:35:28.133839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:35:28.134678 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:35:28.138412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1245/2490] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:35:27 2022] DELETE /identity/v3/projects/95935852764e413c8bbd23c16ddf13a5 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:35:33.864697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:33.866371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:33.954868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f013e7-ed7b-4bef-b595-e4afd6bf099f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:34.021141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:35:34.048788 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc69e-9563-4146-9171-002bcfd56bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:34.050139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc69e-9563-4146-9171-002bcfd56bcc service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:34.050991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc69e-9563-4146-9171-002bcfd56bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:34.052139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc69e-9563-4146-9171-002bcfd56bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:34.060406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-882fc69e-9563-4146-9171-002bcfd56bcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:34.162434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fc69e-9563-4146-9171-002bcfd56bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:34.164234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fc69e-9563-4146-9171-002bcfd56bcc service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:34.175980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1246/2491] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:35:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:35:37.312474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:35:37.313392 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:37.471059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb01ddb-2a96-4a19-b4f4-42e40e16152b admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:35:37.482205 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704cd23-875c-493f-ba2a-e61b436bf79e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:35:37.483396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704cd23-875c-493f-ba2a-e61b436bf79e service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:35:37.483925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704cd23-875c-493f-ba2a-e61b436bf79e service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:35:37.484482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704cd23-875c-493f-ba2a-e61b436bf79e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:35:37.490117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6704cd23-875c-493f-ba2a-e61b436bf79e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:35:37.507284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:35:37.516672 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704cd23-875c-493f-ba2a-e61b436bf79e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:35:37.519981 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704cd23-875c-493f-ba2a-e61b436bf79e service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:35:37.541820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1246/2492] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:35:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:35:44.170872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=34 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:35:47.722180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:06.514926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc69e-9563-4146-9171-002bcfd56bcc service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:06.516653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-882fc69e-9563-4146-9171-002bcfd56bcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:06.529008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc69e-9563-4146-9171-002bcfd56bcc service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:06.533308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b68f6-a114-43ec-825d-ccc617c59838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:06.534041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b68f6-a114-43ec-825d-ccc617c59838 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:06.534386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b68f6-a114-43ec-825d-ccc617c59838 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:06.534703 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b68f6-a114-43ec-825d-ccc617c59838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:06.536679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-003b68f6-a114-43ec-825d-ccc617c59838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:06.623884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b68f6-a114-43ec-825d-ccc617c59838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:06.625476 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b68f6-a114-43ec-825d-ccc617c59838 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:06.637353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1247/2493] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:36:11.674166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704cd23-875c-493f-ba2a-e61b436bf79e service nova] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:11.675235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6704cd23-875c-493f-ba2a-e61b436bf79e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:11.770621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704cd23-875c-493f-ba2a-e61b436bf79e service nova] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:11.775870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:11.776565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:11.776852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:11.777201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] PATH_INFO: `/v3/users/da9978cab3fb4bdda39a2e4718ee2cc7` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:11.778555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=da9978cab3fb4bdda39a2e4718ee2cc7)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:11.780688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:12.090549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da9978cab3fb4bdda39a2e4718ee2cc7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:12.091410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da9978cab3fb4bdda39a2e4718ee2cc7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:12.092310 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da9978cab3fb4bdda39a2e4718ee2cc7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:12.102967 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da9978cab3fb4bdda39a2e4718ee2cc7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:12.122337 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d5755ff4-3a14-4055-a743-0f304421ee3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da9978cab3fb4bdda39a2e4718ee2cc7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:12.219045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1247/2494] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:36:11 2022] DELETE /identity/v3/users/da9978cab3fb4bdda39a2e4718ee2cc7 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:36:12.515134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b68f6-a114-43ec-825d-ccc617c59838 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:12.576287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b68f6-a114-43ec-825d-ccc617c59838 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:12.652443 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:12.653533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:12.653861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:12.654171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] PATH_INFO: `/v3/projects/1b8939e407ff4bdc81191af7f0593cd5` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:12.655794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b8939e407ff4bdc81191af7f0593cd5)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:12.659526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:12.835000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b8939e407ff4bdc81191af7f0593cd5', 'request_id': 'req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:12.840565 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:12.842063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b8939e407ff4bdc81191af7f0593cd5', 'request_id': 'req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:12.927774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b8939e407ff4bdc81191af7f0593cd5', 'request_id': 'req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:12.940851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] The token cache is being invalidate because project 1b8939e407ff4bdc81191af7f0593cd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:36:12.941132 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:12.941798 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b6a861c2-a02d-41ba-b96a-a9d4f85dc701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:12.943468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1248/2495] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:36:12 2022] DELETE /identity/v3/projects/1b8939e407ff4bdc81191af7f0593cd5 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:36:12.949652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082265bc-2001-4a67-88e1-fd05de3d803f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:12.950132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082265bc-2001-4a67-88e1-fd05de3d803f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:12.950364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082265bc-2001-4a67-88e1-fd05de3d803f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:13.014007 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-082265bc-2001-4a67-88e1-fd05de3d803f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:13.015317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-082265bc-2001-4a67-88e1-fd05de3d803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:13.053827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:13.092454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1248/2496] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:13.100575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1ba41-f9ff-49d4-ba9f-381a8c8e6f8f None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:13.100867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1ba41-f9ff-49d4-ba9f-381a8c8e6f8f None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:13.101150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf1ba41-f9ff-49d4-ba9f-381a8c8e6f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:13.121431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-ccf1ba41-f9ff-49d4-ba9f-381a8c8e6f8f None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:13.122672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ccf1ba41-f9ff-49d4-ba9f-381a8c8e6f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:13.130779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1249/2497] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:13.135181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082265bc-2001-4a67-88e1-fd05de3d803f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:13.146308 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082265bc-2001-4a67-88e1-fd05de3d803f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:13.153772 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492b530-4014-47d1-bb0f-ded662d0c6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:13.154535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492b530-4014-47d1-bb0f-ded662d0c6c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:13.154911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492b530-4014-47d1-bb0f-ded662d0c6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:13.155281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492b530-4014-47d1-bb0f-ded662d0c6c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:13.156113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492b530-4014-47d1-bb0f-ded662d0c6c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:36:13.156527 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492b530-4014-47d1-bb0f-ded662d0c6c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:13.158370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492b530-4014-47d1-bb0f-ded662d0c6c8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:13.173128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1249/2498] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:36:13.177685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1ba41-f9ff-49d4-ba9f-381a8c8e6f8f None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:13.187912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf1ba41-f9ff-49d4-ba9f-381a8c8e6f8f None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:13.192684 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddfb53-6932-4431-bd49-62ea5ab5ce37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:13.193307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddfb53-6932-4431-bd49-62ea5ab5ce37 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:13.193585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddfb53-6932-4431-bd49-62ea5ab5ce37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:13.193889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bddfb53-6932-4431-bd49-62ea5ab5ce37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:13.194543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bddfb53-6932-4431-bd49-62ea5ab5ce37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:13.196545 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bddfb53-6932-4431-bd49-62ea5ab5ce37 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:13.360590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1250/2499] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:36:13 2022] POST /identity/v3/projects => generated 403 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:36:13.364799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492b530-4014-47d1-bb0f-ded662d0c6c8 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:13.376136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492b530-4014-47d1-bb0f-ded662d0c6c8 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:13.437330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62036e61-448a-473e-8b32-6b1b57c5d95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:13.437955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62036e61-448a-473e-8b32-6b1b57c5d95a admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:13.438240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62036e61-448a-473e-8b32-6b1b57c5d95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:13.438525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62036e61-448a-473e-8b32-6b1b57c5d95a admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:13.439184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62036e61-448a-473e-8b32-6b1b57c5d95a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:13.441145 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62036e61-448a-473e-8b32-6b1b57c5d95a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:13.659608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1250/2500] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:36:13 2022] POST /identity/v3/users => generated 302 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:36:13.663794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddfb53-6932-4431-bd49-62ea5ab5ce37 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:13.680624 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bddfb53-6932-4431-bd49-62ea5ab5ce37 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:13.686091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c829c-844e-4254-85f2-c133394d6b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:13.686726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c829c-844e-4254-85f2-c133394d6b85 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:13.687011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c829c-844e-4254-85f2-c133394d6b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:13.687290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c829c-844e-4254-85f2-c133394d6b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:13.687987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c829c-844e-4254-85f2-c133394d6b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:13.695351 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c829c-844e-4254-85f2-c133394d6b85 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:13.700785 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1251/2501] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:36:13.704948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62036e61-448a-473e-8b32-6b1b57c5d95a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:13.715521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62036e61-448a-473e-8b32-6b1b57c5d95a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:13.720265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d169e-6df5-4a83-b733-f06d4936a0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:13.721134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d169e-6df5-4a83-b733-f06d4936a0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:13.721520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d169e-6df5-4a83-b733-f06d4936a0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:13.721698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d169e-6df5-4a83-b733-f06d4936a0c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:13.722369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d169e-6df5-4a83-b733-f06d4936a0c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:13.723985 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d169e-6df5-4a83-b733-f06d4936a0c9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:13.741787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1251/2502] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:36:13.743175 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c829c-844e-4254-85f2-c133394d6b85 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:13.755856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c829c-844e-4254-85f2-c133394d6b85 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:13.760501 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63eb88-e16a-4266-b615-55f8fe2a35b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:13.761199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63eb88-e16a-4266-b615-55f8fe2a35b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:13.761477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63eb88-e16a-4266-b615-55f8fe2a35b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:13.761755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63eb88-e16a-4266-b615-55f8fe2a35b7 admin admin] PATH_INFO: `/v3/projects/2e6e164831d64232a814519e01db4535/users/2e431ccf5a8c4bf983d4d01801ff6dd1/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:13.778849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63eb88-e16a-4266-b615-55f8fe2a35b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6e164831d64232a814519e01db4535, user_id=2e431ccf5a8c4bf983d4d01801ff6dd1, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:13.781832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63eb88-e16a-4266-b615-55f8fe2a35b7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:13.905141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1252/2503] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:36:13 2022] PUT /identity/v3/projects/2e6e164831d64232a814519e01db4535/users/2e431ccf5a8c4bf983d4d01801ff6dd1/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:36:13.910596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc32c96-c598-4fd7-b8a7-0d159129c5a8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:13.910806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc32c96-c598-4fd7-b8a7-0d159129c5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:13.911170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc32c96-c598-4fd7-b8a7-0d159129c5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:13.933778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5dc32c96-c598-4fd7-b8a7-0d159129c5a8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:13.935031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5dc32c96-c598-4fd7-b8a7-0d159129c5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:14.004358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1252/2504] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:14.014679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63eb88-e16a-4266-b615-55f8fe2a35b7 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:14.016644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0c63eb88-e16a-4266-b615-55f8fe2a35b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:14.136331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63eb88-e16a-4266-b615-55f8fe2a35b7 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:14.140295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e31574-5a12-4b98-88f8-a1433ca1efa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:14.140922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e31574-5a12-4b98-88f8-a1433ca1efa0 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:14.141199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e31574-5a12-4b98-88f8-a1433ca1efa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:14.141471 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e31574-5a12-4b98-88f8-a1433ca1efa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:14.152459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e31574-5a12-4b98-88f8-a1433ca1efa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:14.154156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e31574-5a12-4b98-88f8-a1433ca1efa0 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:14.165339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1253/2505] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:36:15.783503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89949d2c-6ad4-4fa3-b700-cbce41e01f31 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:15.784071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89949d2c-6ad4-4fa3-b700-cbce41e01f31 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:15.786296 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89949d2c-6ad4-4fa3-b700-cbce41e01f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:15.824272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-89949d2c-6ad4-4fa3-b700-cbce41e01f31 None None] MFA Rules not processed for user `2e431ccf5a8c4bf983d4d01801ff6dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:15.825934 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-89949d2c-6ad4-4fa3-b700-cbce41e01f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:15.874734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:15.909664 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1253/2506] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:36:15 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:15.914698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e31574-5a12-4b98-88f8-a1433ca1efa0 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:15.926220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e31574-5a12-4b98-88f8-a1433ca1efa0 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:15.931654 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3be1de-93d4-4691-a304-3e83f318a05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:15.932386 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3be1de-93d4-4691-a304-3e83f318a05f admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:15.932966 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3be1de-93d4-4691-a304-3e83f318a05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:15.933087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3be1de-93d4-4691-a304-3e83f318a05f admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:15.933793 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3be1de-93d4-4691-a304-3e83f318a05f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:15.935787 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3be1de-93d4-4691-a304-3e83f318a05f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:16.053288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1254/2507] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:36:15 2022] POST /identity/v3/projects => generated 403 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:36:16.058177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89949d2c-6ad4-4fa3-b700-cbce41e01f31 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:16.069388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89949d2c-6ad4-4fa3-b700-cbce41e01f31 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:16.254506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d98847-3092-44e1-8ae5-0c3c3ad194d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:16.255032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d98847-3092-44e1-8ae5-0c3c3ad194d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:16.255397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d98847-3092-44e1-8ae5-0c3c3ad194d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:16.255612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d98847-3092-44e1-8ae5-0c3c3ad194d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:16.256268 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d98847-3092-44e1-8ae5-0c3c3ad194d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:16.258286 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d98847-3092-44e1-8ae5-0c3c3ad194d3 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:16.396662 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1254/2508] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:36:16 2022] POST /identity/v3/users => generated 308 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:36:16.401072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3be1de-93d4-4691-a304-3e83f318a05f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:16.412189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3be1de-93d4-4691-a304-3e83f318a05f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:16.417460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd828c-f14f-4a9d-9a49-27f08e2ba91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:16.418101 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd828c-f14f-4a9d-9a49-27f08e2ba91d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:16.418442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd828c-f14f-4a9d-9a49-27f08e2ba91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:16.418750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd828c-f14f-4a9d-9a49-27f08e2ba91d admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:16.419416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd828c-f14f-4a9d-9a49-27f08e2ba91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:16.421146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd828c-f14f-4a9d-9a49-27f08e2ba91d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:16.434173 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1255/2509] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:36:16.438560 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d98847-3092-44e1-8ae5-0c3c3ad194d3 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:16.452132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d98847-3092-44e1-8ae5-0c3c3ad194d3 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:16.457474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aad498-70f9-4a52-b444-943be8ec7882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:16.458164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aad498-70f9-4a52-b444-943be8ec7882 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:16.458439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aad498-70f9-4a52-b444-943be8ec7882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:16.458710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aad498-70f9-4a52-b444-943be8ec7882 admin admin] PATH_INFO: `/v3/projects/6134ae91422c4d3d8fc9f2f85e5411eb/users/a4ed8e2662d84c2997291a2ab1d87f44/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:16.477213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aad498-70f9-4a52-b444-943be8ec7882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6134ae91422c4d3d8fc9f2f85e5411eb, user_id=a4ed8e2662d84c2997291a2ab1d87f44, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:16.480002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aad498-70f9-4a52-b444-943be8ec7882 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:16.605632 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1255/2510] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:36:16 2022] PUT /identity/v3/projects/6134ae91422c4d3d8fc9f2f85e5411eb/users/a4ed8e2662d84c2997291a2ab1d87f44/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:36:16.609987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd828c-f14f-4a9d-9a49-27f08e2ba91d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:16.621779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd828c-f14f-4a9d-9a49-27f08e2ba91d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:16.630158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:16.694820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fffa8ff-20f0-4e15-be5d-271fe58513f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:16.695905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffa8ff-20f0-4e15-be5d-271fe58513f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:16.695905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffa8ff-20f0-4e15-be5d-271fe58513f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:16.696433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fffa8ff-20f0-4e15-be5d-271fe58513f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:16.697284 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffa8ff-20f0-4e15-be5d-271fe58513f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:16.699506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fffa8ff-20f0-4e15-be5d-271fe58513f2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:16.710763 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1256/2511] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:36:16.715898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aad498-70f9-4a52-b444-943be8ec7882 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:16.727199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aad498-70f9-4a52-b444-943be8ec7882 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:16.732626 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648540b-845c-47b4-b993-21f5d682250d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:16.736871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648540b-845c-47b4-b993-21f5d682250d admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:16.736871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648540b-845c-47b4-b993-21f5d682250d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:16.736871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648540b-845c-47b4-b993-21f5d682250d admin admin] PATH_INFO: `/v3/projects/6134ae91422c4d3d8fc9f2f85e5411eb/users/a4ed8e2662d84c2997291a2ab1d87f44/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:16.736871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648540b-845c-47b4-b993-21f5d682250d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6134ae91422c4d3d8fc9f2f85e5411eb, user_id=a4ed8e2662d84c2997291a2ab1d87f44, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:16.738582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5648540b-845c-47b4-b993-21f5d682250d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:16.750596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1256/2512] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:36:16 2022] PUT /identity/v3/projects/6134ae91422c4d3d8fc9f2f85e5411eb/users/a4ed8e2662d84c2997291a2ab1d87f44/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:36:18.361308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ad266-1884-47d5-93cc-862587c68949 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:18.362010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ad266-1884-47d5-93cc-862587c68949 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:18.362010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ad266-1884-47d5-93cc-862587c68949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:18.394546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-c54ad266-1884-47d5-93cc-862587c68949 None None] MFA Rules not processed for user `a4ed8e2662d84c2997291a2ab1d87f44`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:18.395936 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c54ad266-1884-47d5-93cc-862587c68949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:18.500474 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1257/2513] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:36:18 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:20.129104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648540b-845c-47b4-b993-21f5d682250d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:20.198987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5648540b-845c-47b4-b993-21f5d682250d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:20.203693 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:20.204325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:20.204600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:20.204883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] PATH_INFO: `/v3/users/2e431ccf5a8c4bf983d4d01801ff6dd1` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:20.206021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e431ccf5a8c4bf983d4d01801ff6dd1)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:20.207955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:20.491225 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:20.508501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e431ccf5a8c4bf983d4d01801ff6dd1'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:20.509280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e431ccf5a8c4bf983d4d01801ff6dd1'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:20.509903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e431ccf5a8c4bf983d4d01801ff6dd1'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:20.517722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e431ccf5a8c4bf983d4d01801ff6dd1'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:20.532354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e431ccf5a8c4bf983d4d01801ff6dd1'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:20.676445 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1257/2514] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:36:20 2022] DELETE /identity/v3/users/2e431ccf5a8c4bf983d4d01801ff6dd1 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:36:20.998894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ad266-1884-47d5-93cc-862587c68949 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:21.013231 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ad266-1884-47d5-93cc-862587c68949 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:21.076949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:21.077604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:21.077889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:21.078260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] PATH_INFO: `/v3/projects/2e6e164831d64232a814519e01db4535` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:21.079338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6e164831d64232a814519e01db4535)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:21.081761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:21.230213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:21.243992 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6e164831d64232a814519e01db4535', 'request_id': 'req-a2d47ed0-efbe-4150-b5c2-666f648ba731'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:21.250419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6e164831d64232a814519e01db4535', 'request_id': 'req-a2d47ed0-efbe-4150-b5c2-666f648ba731'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:21.335896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6e164831d64232a814519e01db4535', 'request_id': 'req-a2d47ed0-efbe-4150-b5c2-666f648ba731'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:21.349021 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] The token cache is being invalidate because project 2e6e164831d64232a814519e01db4535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:36:21.349318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:21.350091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:21.351728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1258/2515] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:36:20 2022] DELETE /identity/v3/projects/2e6e164831d64232a814519e01db4535 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:36:21.355969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:21.357557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:21.371590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc4e9bc-0f42-4041-bfa1-6015a691125c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:21.430132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:21.430821 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:21.431129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:21.431410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] PATH_INFO: `/v3/users/a4ed8e2662d84c2997291a2ab1d87f44` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:21.433160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4ed8e2662d84c2997291a2ab1d87f44)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:21.435215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:21.752686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4ed8e2662d84c2997291a2ab1d87f44'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:21.753382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4ed8e2662d84c2997291a2ab1d87f44'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:21.754122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4ed8e2662d84c2997291a2ab1d87f44'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:21.761609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4ed8e2662d84c2997291a2ab1d87f44'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:21.774383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61418494-72dd-4dcb-81ce-8ef26dd94ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4ed8e2662d84c2997291a2ab1d87f44'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:21.874088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1258/2516] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:36:21 2022] DELETE /identity/v3/users/a4ed8e2662d84c2997291a2ab1d87f44 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:36:22.176958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:22.178485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:22.194310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d47ed0-efbe-4150-b5c2-666f648ba731 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:22.260788 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:22.261434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:22.261715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:22.262017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] PATH_INFO: `/v3/projects/6134ae91422c4d3d8fc9f2f85e5411eb` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:22.263176 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6134ae91422c4d3d8fc9f2f85e5411eb)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:22.265171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:22.384112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6134ae91422c4d3d8fc9f2f85e5411eb', 'request_id': 'req-8552489a-a747-4638-ac9e-25c5709c74d0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:22.391260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6134ae91422c4d3d8fc9f2f85e5411eb', 'request_id': 'req-8552489a-a747-4638-ac9e-25c5709c74d0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:22.511157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6134ae91422c4d3d8fc9f2f85e5411eb', 'request_id': 'req-8552489a-a747-4638-ac9e-25c5709c74d0'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:22.523389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] The token cache is being invalidate because project 6134ae91422c4d3d8fc9f2f85e5411eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:36:22.523700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:22.524380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8552489a-a747-4638-ac9e-25c5709c74d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:36:22.526507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1259/2517] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:36:22 2022] DELETE /identity/v3/projects/6134ae91422c4d3d8fc9f2f85e5411eb => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:36:22.531605 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a6ebe-1390-4264-872d-008d6ce2d5f8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:22.532098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a6ebe-1390-4264-872d-008d6ce2d5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:22.533361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a6ebe-1390-4264-872d-008d6ce2d5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:22.551972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-289a6ebe-1390-4264-872d-008d6ce2d5f8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:22.554281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-289a6ebe-1390-4264-872d-008d6ce2d5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:22.579746 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:22.620166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1259/2518] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:22.630666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f396dd-c2ec-455d-b734-c00932846860 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:22.630949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f396dd-c2ec-455d-b734-c00932846860 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:22.631226 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f396dd-c2ec-455d-b734-c00932846860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:22.654337 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-00f396dd-c2ec-455d-b734-c00932846860 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:22.655799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-00f396dd-c2ec-455d-b734-c00932846860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:22.665401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1260/2519] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:36:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:22.670101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a6ebe-1390-4264-872d-008d6ce2d5f8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:22.680233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a6ebe-1390-4264-872d-008d6ce2d5f8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:22.686371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3c907-d066-46b5-9fdf-1025d2fffa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:22.687041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c907-d066-46b5-9fdf-1025d2fffa5d admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:22.687324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c907-d066-46b5-9fdf-1025d2fffa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:22.687634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c907-d066-46b5-9fdf-1025d2fffa5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:22.688312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3c907-d066-46b5-9fdf-1025d2fffa5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:36:22.688623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3c907-d066-46b5-9fdf-1025d2fffa5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:22.690273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3c907-d066-46b5-9fdf-1025d2fffa5d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:22.705774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1260/2520] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:36:22.710271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f396dd-c2ec-455d-b734-c00932846860 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:22.721293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f396dd-c2ec-455d-b734-c00932846860 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:22.731619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3b805-b754-49a9-a219-165ba4f0c7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:22.732262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3b805-b754-49a9-a219-165ba4f0c7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:22.732570 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3b805-b754-49a9-a219-165ba4f0c7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:22.732808 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3b805-b754-49a9-a219-165ba4f0c7f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:22.733442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3b805-b754-49a9-a219-165ba4f0c7f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:22.735977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3b805-b754-49a9-a219-165ba4f0c7f7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:22.911607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1261/2521] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:36:22 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:36:22.916291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3c907-d066-46b5-9fdf-1025d2fffa5d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:22.926483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3c907-d066-46b5-9fdf-1025d2fffa5d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:22.983960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5c700-e5cd-4a15-80c5-818394b57488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:22.984575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5c700-e5cd-4a15-80c5-818394b57488 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:22.984856 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5c700-e5cd-4a15-80c5-818394b57488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:22.985321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5c700-e5cd-4a15-80c5-818394b57488 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:22.985859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5c700-e5cd-4a15-80c5-818394b57488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:22.987732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5c700-e5cd-4a15-80c5-818394b57488 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:23.073939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1261/2522] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:36:22 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:36:23.078252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3b805-b754-49a9-a219-165ba4f0c7f7 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:23.090892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3b805-b754-49a9-a219-165ba4f0c7f7 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:23.096222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af2fb5-7fc4-4411-9c4e-6eeb4168fd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:23.096814 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af2fb5-7fc4-4411-9c4e-6eeb4168fd29 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:23.097220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af2fb5-7fc4-4411-9c4e-6eeb4168fd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:23.097361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af2fb5-7fc4-4411-9c4e-6eeb4168fd29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:23.098015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af2fb5-7fc4-4411-9c4e-6eeb4168fd29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:23.099743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af2fb5-7fc4-4411-9c4e-6eeb4168fd29 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:23.112253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1262/2523] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:36:23.115267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5c700-e5cd-4a15-80c5-818394b57488 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:23.126033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5c700-e5cd-4a15-80c5-818394b57488 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:23.130743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a757855e-59a6-4c8a-b0be-93562f2586aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:23.131355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757855e-59a6-4c8a-b0be-93562f2586aa admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:23.131671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757855e-59a6-4c8a-b0be-93562f2586aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:23.131948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a757855e-59a6-4c8a-b0be-93562f2586aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:23.132598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757855e-59a6-4c8a-b0be-93562f2586aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:23.134186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a757855e-59a6-4c8a-b0be-93562f2586aa admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:23.145732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1262/2524] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:36:23.150088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af2fb5-7fc4-4411-9c4e-6eeb4168fd29 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:23.160626 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af2fb5-7fc4-4411-9c4e-6eeb4168fd29 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:23.165439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e10ed9-a8f1-4d0d-98d8-2c5d8e874c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:23.166099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e10ed9-a8f1-4d0d-98d8-2c5d8e874c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:23.166892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e10ed9-a8f1-4d0d-98d8-2c5d8e874c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:23.167189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e10ed9-a8f1-4d0d-98d8-2c5d8e874c1b admin admin] PATH_INFO: `/v3/projects/dc3ae8e5d8d948fbb5b8702c4fae0d26/users/e9bc5e62bc2541c29d8e044f81e046fa/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:23.170611 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:23.184519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e10ed9-a8f1-4d0d-98d8-2c5d8e874c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3ae8e5d8d948fbb5b8702c4fae0d26, user_id=e9bc5e62bc2541c29d8e044f81e046fa, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:23.187318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e10ed9-a8f1-4d0d-98d8-2c5d8e874c1b admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:23.280027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1263/2525] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:36:23 2022] PUT /identity/v3/projects/dc3ae8e5d8d948fbb5b8702c4fae0d26/users/e9bc5e62bc2541c29d8e044f81e046fa/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:36:23.291780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304036-606b-4ce8-96a3-420a3dd1e576 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:23.292065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304036-606b-4ce8-96a3-420a3dd1e576 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:23.292334 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31304036-606b-4ce8-96a3-420a3dd1e576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:23.322109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-31304036-606b-4ce8-96a3-420a3dd1e576 None None] MFA Rules not processed for user `e9bc5e62bc2541c29d8e044f81e046fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:23.324080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-31304036-606b-4ce8-96a3-420a3dd1e576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:23.393027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1263/2526] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:36:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:23.397842 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e10ed9-a8f1-4d0d-98d8-2c5d8e874c1b admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:23.467931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e10ed9-a8f1-4d0d-98d8-2c5d8e874c1b admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:23.473419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13942f6-28ee-49fb-b95a-3d8bf28bfbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:23.474039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13942f6-28ee-49fb-b95a-3d8bf28bfbb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:23.474326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13942f6-28ee-49fb-b95a-3d8bf28bfbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:23.474608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13942f6-28ee-49fb-b95a-3d8bf28bfbb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:23.475264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13942f6-28ee-49fb-b95a-3d8bf28bfbb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:23.477248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13942f6-28ee-49fb-b95a-3d8bf28bfbb1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:23.571803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1264/2527] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:36:23 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:36:23.576081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304036-606b-4ce8-96a3-420a3dd1e576 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:23.587494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31304036-606b-4ce8-96a3-420a3dd1e576 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:23.609549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:23.654132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333ac4d-cc4d-47dc-9876-99e4cba8a573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:23.654696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333ac4d-cc4d-47dc-9876-99e4cba8a573 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:23.654982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333ac4d-cc4d-47dc-9876-99e4cba8a573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:23.655258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333ac4d-cc4d-47dc-9876-99e4cba8a573 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:23.656210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333ac4d-cc4d-47dc-9876-99e4cba8a573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:23.657891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333ac4d-cc4d-47dc-9876-99e4cba8a573 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:23.756941 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1264/2528] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:36:23 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:36:23.762122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13942f6-28ee-49fb-b95a-3d8bf28bfbb1 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:23.773270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13942f6-28ee-49fb-b95a-3d8bf28bfbb1 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:23.779232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c6708-e16e-4ee7-9eae-061fc2cacb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:23.779903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c6708-e16e-4ee7-9eae-061fc2cacb65 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:23.780182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c6708-e16e-4ee7-9eae-061fc2cacb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:23.780455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c6708-e16e-4ee7-9eae-061fc2cacb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:23.781139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c6708-e16e-4ee7-9eae-061fc2cacb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:23.782933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c6708-e16e-4ee7-9eae-061fc2cacb65 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:23.794727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1265/2529] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:36:23.799337 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333ac4d-cc4d-47dc-9876-99e4cba8a573 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:23.809602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333ac4d-cc4d-47dc-9876-99e4cba8a573 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:23.816988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e211d-e467-4a1f-b771-2d6f6303e7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:23.817701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e211d-e467-4a1f-b771-2d6f6303e7fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:23.817980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e211d-e467-4a1f-b771-2d6f6303e7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:23.818252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e211d-e467-4a1f-b771-2d6f6303e7fc admin admin] PATH_INFO: `/v3/projects/69c917ec65b047699f647e9e19f4e3a0/users/f7d22d145826401287d795d5d93ae298/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:23.834642 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e211d-e467-4a1f-b771-2d6f6303e7fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c917ec65b047699f647e9e19f4e3a0, user_id=f7d22d145826401287d795d5d93ae298, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:23.837330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e211d-e467-4a1f-b771-2d6f6303e7fc admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:23.943590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1265/2530] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:36:23 2022] PUT /identity/v3/projects/69c917ec65b047699f647e9e19f4e3a0/users/f7d22d145826401287d795d5d93ae298/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:36:23.947732 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c6708-e16e-4ee7-9eae-061fc2cacb65 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:23.958519 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c6708-e16e-4ee7-9eae-061fc2cacb65 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:24.023366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692760d-fb99-4c8f-aaf5-90f12371391d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:24.024024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692760d-fb99-4c8f-aaf5-90f12371391d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:24.024302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692760d-fb99-4c8f-aaf5-90f12371391d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:24.024577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692760d-fb99-4c8f-aaf5-90f12371391d admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:24.025239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692760d-fb99-4c8f-aaf5-90f12371391d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:24.026903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692760d-fb99-4c8f-aaf5-90f12371391d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:24.038309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1266/2531] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:36:24.043338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e211d-e467-4a1f-b771-2d6f6303e7fc admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:24.054438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e211d-e467-4a1f-b771-2d6f6303e7fc admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:24.060059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a79fc-e901-408b-b536-69271fdc3b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:24.060875 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a79fc-e901-408b-b536-69271fdc3b8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:24.060987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a79fc-e901-408b-b536-69271fdc3b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:24.061261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a79fc-e901-408b-b536-69271fdc3b8d admin admin] PATH_INFO: `/v3/projects/69c917ec65b047699f647e9e19f4e3a0/users/f7d22d145826401287d795d5d93ae298/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:24.063309 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a79fc-e901-408b-b536-69271fdc3b8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c917ec65b047699f647e9e19f4e3a0, user_id=f7d22d145826401287d795d5d93ae298, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:24.065998 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a79fc-e901-408b-b536-69271fdc3b8d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:24.080133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1266/2532] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:36:24 2022] PUT /identity/v3/projects/69c917ec65b047699f647e9e19f4e3a0/users/f7d22d145826401287d795d5d93ae298/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:36:24.091052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18091b-754e-4808-8a2d-f9fb245a0e51 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:24.091267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18091b-754e-4808-8a2d-f9fb245a0e51 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:24.091587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18091b-754e-4808-8a2d-f9fb245a0e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:24.122327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-6f18091b-754e-4808-8a2d-f9fb245a0e51 None None] MFA Rules not processed for user `f7d22d145826401287d795d5d93ae298`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:24.123604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6f18091b-754e-4808-8a2d-f9fb245a0e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:24.169034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:24.194754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1267/2533] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:36:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:24.206242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a79fc-e901-408b-b536-69271fdc3b8d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:24.208250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e93a79fc-e901-408b-b536-69271fdc3b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:24.298092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a79fc-e901-408b-b536-69271fdc3b8d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:24.301240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29f9f6-6810-4270-8e01-5321e9508344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:24.301868 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29f9f6-6810-4270-8e01-5321e9508344 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:24.302150 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29f9f6-6810-4270-8e01-5321e9508344 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:24.302427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29f9f6-6810-4270-8e01-5321e9508344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:24.312365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29f9f6-6810-4270-8e01-5321e9508344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:24.313895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29f9f6-6810-4270-8e01-5321e9508344 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:24.324182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1267/2534] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:36:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:36:24.559390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18091b-754e-4808-8a2d-f9fb245a0e51 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:24.569067 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18091b-754e-4808-8a2d-f9fb245a0e51 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:24.573315 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949900d-6135-49d3-9ee7-b06fc3e7aee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:24.573921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949900d-6135-49d3-9ee7-b06fc3e7aee8 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:24.574200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949900d-6135-49d3-9ee7-b06fc3e7aee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:24.574468 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3949900d-6135-49d3-9ee7-b06fc3e7aee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:24.649491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949900d-6135-49d3-9ee7-b06fc3e7aee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:24.651603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3949900d-6135-49d3-9ee7-b06fc3e7aee8 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:24.661540 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1268/2535] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:36:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:36:34.319504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:34.656700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=37 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:35.088128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fa2c1-140b-4deb-b8b4-bda85d56fe39 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:35.088952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fa2c1-140b-4deb-b8b4-bda85d56fe39 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:35.089226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fa2c1-140b-4deb-b8b4-bda85d56fe39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:35.119370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-260fa2c1-140b-4deb-b8b4-bda85d56fe39 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:35.121033 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-260fa2c1-140b-4deb-b8b4-bda85d56fe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:35.179435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1268/2536] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:36:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:35.358666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949900d-6135-49d3-9ee7-b06fc3e7aee8 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:35.360208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3949900d-6135-49d3-9ee7-b06fc3e7aee8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:35.451321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3949900d-6135-49d3-9ee7-b06fc3e7aee8 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:35.457510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a968e-5c0e-4244-ad9c-74dcc0fef1d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:35.458152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a968e-5c0e-4244-ad9c-74dcc0fef1d8 service swift] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:35.458430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a968e-5c0e-4244-ad9c-74dcc0fef1d8 service swift] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:35.458707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a968e-5c0e-4244-ad9c-74dcc0fef1d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:35.468150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a968e-5c0e-4244-ad9c-74dcc0fef1d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:35.470552 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a968e-5c0e-4244-ad9c-74dcc0fef1d8 service swift] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:35.480995 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1269/2537] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:36:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 01:36:37.530820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fa2c1-140b-4deb-b8b4-bda85d56fe39 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:37.533134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-260fa2c1-140b-4deb-b8b4-bda85d56fe39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:37.630116 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fa2c1-140b-4deb-b8b4-bda85d56fe39 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:37.640226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f454c-b486-4938-b52d-20661b2d7f95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:37.640858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f454c-b486-4938-b52d-20661b2d7f95 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:37.641213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f454c-b486-4938-b52d-20661b2d7f95 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:37.641581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f454c-b486-4938-b52d-20661b2d7f95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:37.643968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e56f454c-b486-4938-b52d-20661b2d7f95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:37.680047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f454c-b486-4938-b52d-20661b2d7f95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:37.682620 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f454c-b486-4938-b52d-20661b2d7f95 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:37.699110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1269/2538] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:36:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:36:45.477458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=32 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:47.692135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=63 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:36:56.601884 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a968e-5c0e-4244-ad9c-74dcc0fef1d8 service swift] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:56.604026 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d77a968e-5c0e-4244-ad9c-74dcc0fef1d8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:56.662452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a968e-5c0e-4244-ad9c-74dcc0fef1d8 service swift] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:56.666726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98c56b-9424-40ae-b169-107b1fa0973f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:56.667329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c56b-9424-40ae-b169-107b1fa0973f service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:56.667644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c56b-9424-40ae-b169-107b1fa0973f service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:56.667933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c56b-9424-40ae-b169-107b1fa0973f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:56.669762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1a98c56b-9424-40ae-b169-107b1fa0973f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:56.680551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98c56b-9424-40ae-b169-107b1fa0973f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:56.682098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98c56b-9424-40ae-b169-107b1fa0973f service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:56.690817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1270/2539] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:36:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:36:57.203090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8476-756e-400b-aa88-2a7bbb48c1fa None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:57.203702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8476-756e-400b-aa88-2a7bbb48c1fa None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:57.204128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dd8476-756e-400b-aa88-2a7bbb48c1fa None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:57.206675 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1270/2540] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:36:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:36:57.213402 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded849e-750c-4682-a7db-26a95a8727fb None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:57.213849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded849e-750c-4682-a7db-26a95a8727fb None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:57.214041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded849e-750c-4682-a7db-26a95a8727fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:57.234676 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8ded849e-750c-4682-a7db-26a95a8727fb None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:57.236340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8ded849e-750c-4682-a7db-26a95a8727fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:57.246918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1271/2541] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:36:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:57.261450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd8476-756e-400b-aa88-2a7bbb48c1fa None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:57.265461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b9dd8476-756e-400b-aa88-2a7bbb48c1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:57.335000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dd8476-756e-400b-aa88-2a7bbb48c1fa None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:57.338793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e69a88-f28e-452d-b75b-72b087b90782 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f10c258fe5f45ccab1669fa32ed6029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:57.339414 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e69a88-f28e-452d-b75b-72b087b90782 service glance] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:57.339804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e69a88-f28e-452d-b75b-72b087b90782 service glance] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:57.340081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e69a88-f28e-452d-b75b-72b087b90782 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:57.351818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e69a88-f28e-452d-b75b-72b087b90782 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:57.353195 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e69a88-f28e-452d-b75b-72b087b90782 service glance] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:57.365512 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1271/2542] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:36:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:36:57.586383 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185d16d-3426-4e4f-af40-e4421c5459b0 None None] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:57.586850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185d16d-3426-4e4f-af40-e4421c5459b0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:57.587253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f185d16d-3426-4e4f-af40-e4421c5459b0 None None] PATH_INFO: `/` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:57.589692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1272/2543] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:36:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:36:57.596765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0cd0a-f78e-4106-be91-dca7aa2d6f6b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:57.597080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0cd0a-f78e-4106-be91-dca7aa2d6f6b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:57.599034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0cd0a-f78e-4106-be91-dca7aa2d6f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:57.620416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3df0cd0a-f78e-4106-be91-dca7aa2d6f6b None None] MFA Rules not processed for user `3b0a7b4f607d4753bf142e1420bcd987`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:57.621674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3df0cd0a-f78e-4106-be91-dca7aa2d6f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:57.628814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1272/2544] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:36:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:36:57.645464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185d16d-3426-4e4f-af40-e4421c5459b0 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:57.647901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f185d16d-3426-4e4f-af40-e4421c5459b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:57.663336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f185d16d-3426-4e4f-af40-e4421c5459b0 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:57.667771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f17d1-91d4-4584-a1ae-a3142df1d150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:57.668432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f17d1-91d4-4584-a1ae-a3142df1d150 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:57.668710 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f17d1-91d4-4584-a1ae-a3142df1d150 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:57.668992 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f17d1-91d4-4584-a1ae-a3142df1d150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:57.677774 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f17d1-91d4-4584-a1ae-a3142df1d150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:57.679251 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f17d1-91d4-4584-a1ae-a3142df1d150 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:57.688742 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1273/2545] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:36:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:36:57.779595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b9b76-0198-47bc-8cd4-a25b236170fd None None] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:57.779883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b9b76-0198-47bc-8cd4-a25b236170fd None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:57.780158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b9b76-0198-47bc-8cd4-a25b236170fd None None] PATH_INFO: `/` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:57.781562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1273/2546] 10.222.0.64 () {54 vars in 921 bytes} [Fri Sep 23 01:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 23 01:36:57.787962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941e4ab-2854-4f00-95f5-c690cc12ee33 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:57.788244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941e4ab-2854-4f00-95f5-c690cc12ee33 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:57.788517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941e4ab-2854-4f00-95f5-c690cc12ee33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:57.810815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f941e4ab-2854-4f00-95f5-c690cc12ee33 None None] MFA Rules not processed for user `1c4da5b2130d4fa990adb72f0f6ec2fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:36:57.812099 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f941e4ab-2854-4f00-95f5-c690cc12ee33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:36:57.822040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1274/2547] 10.222.0.64 () {58 vars in 1018 bytes} [Fri Sep 23 01:36:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:36:57.833756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b9b76-0198-47bc-8cd4-a25b236170fd None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:36:57.847896 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b9b76-0198-47bc-8cd4-a25b236170fd None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:36:57.852597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f6608-df56-4261-9e48-9a243d8b34f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:36:57.853878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f6608-df56-4261-9e48-9a243d8b34f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:36:57.854301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f6608-df56-4261-9e48-9a243d8b34f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:36:57.854586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3f6608-df56-4261-9e48-9a243d8b34f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:36:57.866240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f6608-df56-4261-9e48-9a243d8b34f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:36:57.867763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3f6608-df56-4261-9e48-9a243d8b34f1 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:36:57.880257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1274/2548] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:36:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:02.215194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4003c00-9797-4691-b021-4f66e02ad872 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:02.215194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4003c00-9797-4691-b021-4f66e02ad872 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:02.215194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4003c00-9797-4691-b021-4f66e02ad872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:02.243812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a4003c00-9797-4691-b021-4f66e02ad872 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:02.243812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a4003c00-9797-4691-b021-4f66e02ad872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:02.243812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1275/2549] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:37:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:02.257465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f6608-df56-4261-9e48-9a243d8b34f1 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:02.269587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3f6608-df56-4261-9e48-9a243d8b34f1 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:02.277342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e106ae-6595-4fec-b955-8ca53f639e16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:02.278440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e106ae-6595-4fec-b955-8ca53f639e16 service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:02.280523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e106ae-6595-4fec-b955-8ca53f639e16 service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:02.280523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e106ae-6595-4fec-b955-8ca53f639e16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:02.292888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e106ae-6595-4fec-b955-8ca53f639e16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:02.295266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e106ae-6595-4fec-b955-8ca53f639e16 service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:02.311171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1275/2550] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:37:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:37:12.227200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=41 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:12.303458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=41 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:20.903360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4003c00-9797-4691-b021-4f66e02ad872 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:21.017888 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4003c00-9797-4691-b021-4f66e02ad872 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:21.023397 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:21.027371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:21.027371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:21.027371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] PATH_INFO: `/v3/users/e9bc5e62bc2541c29d8e044f81e046fa` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:21.027371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9bc5e62bc2541c29d8e044f81e046fa)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:21.029178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:21.256810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9bc5e62bc2541c29d8e044f81e046fa'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:21.338295 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6dab76-5024-467a-9edd-73643fef8096 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:21.339264 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6dab76-5024-467a-9edd-73643fef8096 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:21.339536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6dab76-5024-467a-9edd-73643fef8096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:21.374654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-be6dab76-5024-467a-9edd-73643fef8096 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:21.374654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-be6dab76-5024-467a-9edd-73643fef8096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:21.386524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1276/2551] 10.222.0.64 () {58 vars in 967 bytes} [Fri Sep 23 01:37:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:21.403663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6dab76-5024-467a-9edd-73643fef8096 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:21.418332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6dab76-5024-467a-9edd-73643fef8096 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:21.424141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171baf8-c500-47eb-945b-d902f48b6835 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:21.424676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171baf8-c500-47eb-945b-d902f48b6835 service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:21.424958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171baf8-c500-47eb-945b-d902f48b6835 service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:21.425307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171baf8-c500-47eb-945b-d902f48b6835 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:21.434926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171baf8-c500-47eb-945b-d902f48b6835 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:21.436631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171baf8-c500-47eb-945b-d902f48b6835 service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:21.444945 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9bc5e62bc2541c29d8e044f81e046fa'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:21.445667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9bc5e62bc2541c29d8e044f81e046fa'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:21.454618 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1277/2552] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:37:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 01:37:21.466963 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9bc5e62bc2541c29d8e044f81e046fa'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:21.467817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9bc5e62bc2541c29d8e044f81e046fa'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:21.484922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1276/2553] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:20 2022] DELETE /identity/v3/users/e9bc5e62bc2541c29d8e044f81e046fa => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:21.489424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe91342-7d46-4289-82ee-1a32dd3e69f5 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:21.492004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe91342-7d46-4289-82ee-1a32dd3e69f5 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:21.492953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe91342-7d46-4289-82ee-1a32dd3e69f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:21.528522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-5fe91342-7d46-4289-82ee-1a32dd3e69f5 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:21.529813 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5fe91342-7d46-4289-82ee-1a32dd3e69f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:21.598057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1278/2554] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:21.609574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:21.612490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:21.714903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736afc5-df1e-40a3-9fbb-34c252f9ab63 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:21.719366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8b113-fade-4137-a3e9-831a59ce14f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:21.720135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8b113-fade-4137-a3e9-831a59ce14f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:21.720530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8b113-fade-4137-a3e9-831a59ce14f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:21.720994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8b113-fade-4137-a3e9-831a59ce14f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:21.732845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8b113-fade-4137-a3e9-831a59ce14f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:21.734577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc8b113-fade-4137-a3e9-831a59ce14f7 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:21.746096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1277/2555] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:22.336728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe91342-7d46-4289-82ee-1a32dd3e69f5 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:22.338699 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5fe91342-7d46-4289-82ee-1a32dd3e69f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:22.359052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe91342-7d46-4289-82ee-1a32dd3e69f5 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:22.366465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:22.367477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:22.369006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:22.369006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] PATH_INFO: `/v3/projects/dc3ae8e5d8d948fbb5b8702c4fae0d26` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:22.371381 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc3ae8e5d8d948fbb5b8702c4fae0d26)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:22.379280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:22.589710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc3ae8e5d8d948fbb5b8702c4fae0d26', 'request_id': 'req-3e8c1b38-9674-4c0b-aac0-a898b58f295a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:22.591965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc3ae8e5d8d948fbb5b8702c4fae0d26', 'request_id': 'req-3e8c1b38-9674-4c0b-aac0-a898b58f295a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:22.690681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc3ae8e5d8d948fbb5b8702c4fae0d26', 'request_id': 'req-3e8c1b38-9674-4c0b-aac0-a898b58f295a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:22.706946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] The token cache is being invalidate because project dc3ae8e5d8d948fbb5b8702c4fae0d26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:22.707303 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:22.707951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:22.709504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1279/2556] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:22 2022] DELETE /identity/v3/projects/dc3ae8e5d8d948fbb5b8702c4fae0d26 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:22.714597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8b113-fade-4137-a3e9-831a59ce14f7 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:22.725712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8b113-fade-4137-a3e9-831a59ce14f7 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:22.750260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:22.805905 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:22.806846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:22.807195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:22.807598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] PATH_INFO: `/v3/users/f7d22d145826401287d795d5d93ae298` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:22.809253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7d22d145826401287d795d5d93ae298)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:22.812605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:23.105709 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7d22d145826401287d795d5d93ae298'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:23.243576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7d22d145826401287d795d5d93ae298'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:23.244297 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7d22d145826401287d795d5d93ae298'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:23.261769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7d22d145826401287d795d5d93ae298'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:23.262712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1e6d6112-cb8d-40d0-b950-6bee983bb504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7d22d145826401287d795d5d93ae298'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:23.287607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1278/2557] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:22 2022] DELETE /identity/v3/users/f7d22d145826401287d795d5d93ae298 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:23.946209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:23.947853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:23.974631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c1b38-9674-4c0b-aac0-a898b58f295a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:24.089859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:24.090536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:24.090823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:24.091099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] PATH_INFO: `/v3/projects/69c917ec65b047699f647e9e19f4e3a0` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:24.093958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69c917ec65b047699f647e9e19f4e3a0)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:24.096980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:24.106638 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:24.269151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69c917ec65b047699f647e9e19f4e3a0', 'request_id': 'req-4f2fce19-bc40-4354-9973-3fe257baee73'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:24.281776 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69c917ec65b047699f647e9e19f4e3a0', 'request_id': 'req-4f2fce19-bc40-4354-9973-3fe257baee73'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:24.375871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69c917ec65b047699f647e9e19f4e3a0', 'request_id': 'req-4f2fce19-bc40-4354-9973-3fe257baee73'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:24.397802 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] The token cache is being invalidate because project 69c917ec65b047699f647e9e19f4e3a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:24.398276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:24.399347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4f2fce19-bc40-4354-9973-3fe257baee73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:24.402352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1280/2558] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:23 2022] DELETE /identity/v3/projects/69c917ec65b047699f647e9e19f4e3a0 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:24.414204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51705c-6769-4a66-b0f8-bf941072a477 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:24.418206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51705c-6769-4a66-b0f8-bf941072a477 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:24.418206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51705c-6769-4a66-b0f8-bf941072a477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:24.439013 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-1c51705c-6769-4a66-b0f8-bf941072a477 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:24.440416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1c51705c-6769-4a66-b0f8-bf941072a477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:24.519039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1279/2559] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:24.532936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf54c9-ef60-453a-8529-129c3fa23256 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:24.533340 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf54c9-ef60-453a-8529-129c3fa23256 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:24.533830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf54c9-ef60-453a-8529-129c3fa23256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:24.568569 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-10bf54c9-ef60-453a-8529-129c3fa23256 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:24.570769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-10bf54c9-ef60-453a-8529-129c3fa23256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:24.585648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1281/2560] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:24.616913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51705c-6769-4a66-b0f8-bf941072a477 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:24.630023 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c51705c-6769-4a66-b0f8-bf941072a477 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:24.635273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d371f-180b-4bc6-ae19-eb891bfcb173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:24.635989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d371f-180b-4bc6-ae19-eb891bfcb173 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:24.636296 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d371f-180b-4bc6-ae19-eb891bfcb173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:24.636575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d371f-180b-4bc6-ae19-eb891bfcb173 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:24.637240 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d371f-180b-4bc6-ae19-eb891bfcb173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:37:24.637574 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d371f-180b-4bc6-ae19-eb891bfcb173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:24.639240 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d371f-180b-4bc6-ae19-eb891bfcb173 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:24.653329 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1280/2561] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:37:24.659096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf54c9-ef60-453a-8529-129c3fa23256 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:24.673003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf54c9-ef60-453a-8529-129c3fa23256 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:24.679043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277f104-3914-4cc1-8de1-1a47e10cd565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:24.679845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277f104-3914-4cc1-8de1-1a47e10cd565 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:24.680136 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277f104-3914-4cc1-8de1-1a47e10cd565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:24.680498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277f104-3914-4cc1-8de1-1a47e10cd565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:24.681315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277f104-3914-4cc1-8de1-1a47e10cd565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:24.684447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277f104-3914-4cc1-8de1-1a47e10cd565 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:24.840701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1282/2562] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:24 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:37:24.846061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d371f-180b-4bc6-ae19-eb891bfcb173 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:24.858854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d371f-180b-4bc6-ae19-eb891bfcb173 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:24.905614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:24.953922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b9d5b-ad87-435a-86a0-0106218d49ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:24.954612 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b9d5b-ad87-435a-86a0-0106218d49ce admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:24.954870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b9d5b-ad87-435a-86a0-0106218d49ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:24.955143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b9d5b-ad87-435a-86a0-0106218d49ce admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:24.955856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b9d5b-ad87-435a-86a0-0106218d49ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:24.957961 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b9d5b-ad87-435a-86a0-0106218d49ce admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:25.079162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1281/2563] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:24 2022] POST /identity/v3/users => generated 283 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:37:25.084320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277f104-3914-4cc1-8de1-1a47e10cd565 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:25.096370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277f104-3914-4cc1-8de1-1a47e10cd565 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:25.102485 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31df22-2aa4-4c15-ba7e-0571f79aba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:25.103159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31df22-2aa4-4c15-ba7e-0571f79aba74 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:25.103823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31df22-2aa4-4c15-ba7e-0571f79aba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:25.104202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31df22-2aa4-4c15-ba7e-0571f79aba74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:25.104949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31df22-2aa4-4c15-ba7e-0571f79aba74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:25.106884 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31df22-2aa4-4c15-ba7e-0571f79aba74 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:25.119918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1283/2564] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:37:25.124200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b9d5b-ad87-435a-86a0-0106218d49ce admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:25.136139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b9d5b-ad87-435a-86a0-0106218d49ce admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:25.141891 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbf275-a66f-486b-a1f9-db4dbaeebe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:25.142567 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf275-a66f-486b-a1f9-db4dbaeebe5a admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:25.142849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf275-a66f-486b-a1f9-db4dbaeebe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:25.143122 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf275-a66f-486b-a1f9-db4dbaeebe5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:25.143809 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbf275-a66f-486b-a1f9-db4dbaeebe5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:25.145535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbf275-a66f-486b-a1f9-db4dbaeebe5a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:25.157031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1282/2565] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:25.161972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31df22-2aa4-4c15-ba7e-0571f79aba74 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:25.173683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31df22-2aa4-4c15-ba7e-0571f79aba74 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:25.178790 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064795f-b654-4717-b1a8-22c9cce52535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:25.179532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064795f-b654-4717-b1a8-22c9cce52535 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:25.179843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064795f-b654-4717-b1a8-22c9cce52535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:25.180120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064795f-b654-4717-b1a8-22c9cce52535 admin admin] PATH_INFO: `/v3/projects/069a6d65c8f543c489870965355a8214/users/89bd12a289484b9594c29166e26c5ef5/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:25.205391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064795f-b654-4717-b1a8-22c9cce52535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=069a6d65c8f543c489870965355a8214, user_id=89bd12a289484b9594c29166e26c5ef5, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:25.211213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8064795f-b654-4717-b1a8-22c9cce52535 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:25.305565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1284/2566] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:25 2022] PUT /identity/v3/projects/069a6d65c8f543c489870965355a8214/users/89bd12a289484b9594c29166e26c5ef5/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:25.321881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c26b43-46b2-4b8a-8320-f707634d8c58 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:25.322433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c26b43-46b2-4b8a-8320-f707634d8c58 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:25.323110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c26b43-46b2-4b8a-8320-f707634d8c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:25.361489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-27c26b43-46b2-4b8a-8320-f707634d8c58 None None] MFA Rules not processed for user `89bd12a289484b9594c29166e26c5ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:25.363017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-27c26b43-46b2-4b8a-8320-f707634d8c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:25.475595 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1283/2567] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:25.476516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064795f-b654-4717-b1a8-22c9cce52535 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:25.563302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c26b43-46b2-4b8a-8320-f707634d8c58 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:25.564837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-27c26b43-46b2-4b8a-8320-f707634d8c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:25.588156 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064795f-b654-4717-b1a8-22c9cce52535 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:25.596976 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30336d68-e4ea-424d-91da-867afad4141f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:25.598078 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30336d68-e4ea-424d-91da-867afad4141f admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:25.598599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30336d68-e4ea-424d-91da-867afad4141f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:25.599086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30336d68-e4ea-424d-91da-867afad4141f admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:25.600201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30336d68-e4ea-424d-91da-867afad4141f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:25.604038 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30336d68-e4ea-424d-91da-867afad4141f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:25.648464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c26b43-46b2-4b8a-8320-f707634d8c58 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:25.652244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8e06e-5f1f-4c42-944e-5465c8aa1be2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:25.652886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8e06e-5f1f-4c42-944e-5465c8aa1be2 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:25.653169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8e06e-5f1f-4c42-944e-5465c8aa1be2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:25.653449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8e06e-5f1f-4c42-944e-5465c8aa1be2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:25.655910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-31b8e06e-5f1f-4c42-944e-5465c8aa1be2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:25.670256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8e06e-5f1f-4c42-944e-5465c8aa1be2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:25.671773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8e06e-5f1f-4c42-944e-5465c8aa1be2 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:25.687047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1284/2568] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:37:25.715506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1285/2569] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:25 2022] POST /identity/v3/projects => generated 365 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:25.720362 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8e06e-5f1f-4c42-944e-5465c8aa1be2 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:25.731399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8e06e-5f1f-4c42-944e-5465c8aa1be2 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:25.784368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:25.813038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ba485-0f33-47c1-8ff8-bb312b56d233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:25.813794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ba485-0f33-47c1-8ff8-bb312b56d233 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:25.814150 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ba485-0f33-47c1-8ff8-bb312b56d233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:25.814439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ba485-0f33-47c1-8ff8-bb312b56d233 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:25.815108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ba485-0f33-47c1-8ff8-bb312b56d233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:25.819799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ba485-0f33-47c1-8ff8-bb312b56d233 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:25.982360 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1285/2570] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:25 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:37:25.987312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30336d68-e4ea-424d-91da-867afad4141f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:25.999233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30336d68-e4ea-424d-91da-867afad4141f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:26.004502 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782f7ec-d024-4b47-8bfb-8ff4d35ca8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:26.005107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782f7ec-d024-4b47-8bfb-8ff4d35ca8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:26.005400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782f7ec-d024-4b47-8bfb-8ff4d35ca8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:26.005673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f782f7ec-d024-4b47-8bfb-8ff4d35ca8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:26.006345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782f7ec-d024-4b47-8bfb-8ff4d35ca8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:26.008475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f782f7ec-d024-4b47-8bfb-8ff4d35ca8f5 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:26.020738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1286/2571] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:26.025778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ba485-0f33-47c1-8ff8-bb312b56d233 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:26.045503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ba485-0f33-47c1-8ff8-bb312b56d233 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:26.053321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa83e2-aef6-41a3-9054-a212cd132c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:26.054463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa83e2-aef6-41a3-9054-a212cd132c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:26.054966 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa83e2-aef6-41a3-9054-a212cd132c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:26.055464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa83e2-aef6-41a3-9054-a212cd132c59 admin admin] PATH_INFO: `/v3/projects/dc47ca5f143044a0b6ff075f57efac61/users/b8bfd6e0b1c440169ea0698f9c3e3dd0/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:26.095657 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa83e2-aef6-41a3-9054-a212cd132c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc47ca5f143044a0b6ff075f57efac61, user_id=b8bfd6e0b1c440169ea0698f9c3e3dd0, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:26.099327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa83e2-aef6-41a3-9054-a212cd132c59 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:26.200902 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1286/2572] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:26 2022] PUT /identity/v3/projects/dc47ca5f143044a0b6ff075f57efac61/users/b8bfd6e0b1c440169ea0698f9c3e3dd0/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:26.206281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782f7ec-d024-4b47-8bfb-8ff4d35ca8f5 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:26.221307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:26.230003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f782f7ec-d024-4b47-8bfb-8ff4d35ca8f5 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:26.347684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443acc1e-4d07-409e-a13d-ac95ba7a6501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:26.347684 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443acc1e-4d07-409e-a13d-ac95ba7a6501 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:26.348169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443acc1e-4d07-409e-a13d-ac95ba7a6501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:26.348169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443acc1e-4d07-409e-a13d-ac95ba7a6501 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:26.349211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443acc1e-4d07-409e-a13d-ac95ba7a6501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:26.352515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443acc1e-4d07-409e-a13d-ac95ba7a6501 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:26.373401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1287/2573] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:26.380097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa83e2-aef6-41a3-9054-a212cd132c59 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:26.396264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa83e2-aef6-41a3-9054-a212cd132c59 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:26.402621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b201c-ea1d-4895-9758-25ca5d786c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:26.404047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b201c-ea1d-4895-9758-25ca5d786c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:26.404522 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b201c-ea1d-4895-9758-25ca5d786c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:26.404867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b201c-ea1d-4895-9758-25ca5d786c0a admin admin] PATH_INFO: `/v3/projects/dc47ca5f143044a0b6ff075f57efac61/users/b8bfd6e0b1c440169ea0698f9c3e3dd0/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:26.407529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b201c-ea1d-4895-9758-25ca5d786c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc47ca5f143044a0b6ff075f57efac61, user_id=b8bfd6e0b1c440169ea0698f9c3e3dd0, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:26.410318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b201c-ea1d-4895-9758-25ca5d786c0a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:26.424729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1287/2574] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:26 2022] PUT /identity/v3/projects/dc47ca5f143044a0b6ff075f57efac61/users/b8bfd6e0b1c440169ea0698f9c3e3dd0/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:26.439037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bfb9-d5fb-483f-9bee-09e80449d605 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:26.439370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bfb9-d5fb-483f-9bee-09e80449d605 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:26.439701 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bfb9-d5fb-483f-9bee-09e80449d605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:26.493773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-09e7bfb9-d5fb-483f-9bee-09e80449d605 None None] MFA Rules not processed for user `b8bfd6e0b1c440169ea0698f9c3e3dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:26.495194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-09e7bfb9-d5fb-483f-9bee-09e80449d605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:26.604355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1288/2575] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:26.623145 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b201c-ea1d-4895-9758-25ca5d786c0a admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:26.625527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-906b201c-ea1d-4895-9758-25ca5d786c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:26.720710 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b201c-ea1d-4895-9758-25ca5d786c0a admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:26.726268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60db34-7f0b-4064-a453-35be0cfcebea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:26.727044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60db34-7f0b-4064-a453-35be0cfcebea service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:26.727409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60db34-7f0b-4064-a453-35be0cfcebea service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:26.727824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60db34-7f0b-4064-a453-35be0cfcebea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:26.739322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60db34-7f0b-4064-a453-35be0cfcebea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:26.741208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60db34-7f0b-4064-a453-35be0cfcebea service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:26.752727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1288/2576] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:27.038739 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7bfb9-d5fb-483f-9bee-09e80449d605 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:27.050808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e7bfb9-d5fb-483f-9bee-09e80449d605 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:27.055589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d354a-500a-49b7-814a-3808a31855aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:27.058027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d354a-500a-49b7-814a-3808a31855aa service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:27.058027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d354a-500a-49b7-814a-3808a31855aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:27.058027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d354a-500a-49b7-814a-3808a31855aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:27.133527 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d354a-500a-49b7-814a-3808a31855aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:27.136244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d354a-500a-49b7-814a-3808a31855aa service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:27.151110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1289/2577] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:30.357727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60db34-7f0b-4064-a453-35be0cfcebea service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:30.360405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5e60db34-7f0b-4064-a453-35be0cfcebea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:30.448432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60db34-7f0b-4064-a453-35be0cfcebea service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:30.454861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d86f8-d81f-4741-8aaf-bd4196f5086a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:30.455562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d86f8-d81f-4741-8aaf-bd4196f5086a service placement] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:30.455849 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d86f8-d81f-4741-8aaf-bd4196f5086a service placement] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:30.456631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d86f8-d81f-4741-8aaf-bd4196f5086a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:30.458819 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-613d86f8-d81f-4741-8aaf-bd4196f5086a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:30.472881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d86f8-d81f-4741-8aaf-bd4196f5086a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:30.474659 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d86f8-d81f-4741-8aaf-bd4196f5086a service placement] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:30.486493 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1289/2578] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:30.572719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d354a-500a-49b7-814a-3808a31855aa service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:30.574204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b85d354a-500a-49b7-814a-3808a31855aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:30.608828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:30.656442 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d354a-500a-49b7-814a-3808a31855aa service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:30.661634 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:30.662286 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:30.662621 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:30.662881 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] PATH_INFO: `/v3/users/1633ea35eda34ffe884140eaab1ae909` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:30.664141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1633ea35eda34ffe884140eaab1ae909)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:30.666657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:30.875092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1633ea35eda34ffe884140eaab1ae909'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:30.875898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1633ea35eda34ffe884140eaab1ae909'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:30.876478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1633ea35eda34ffe884140eaab1ae909'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:30.885025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1633ea35eda34ffe884140eaab1ae909'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:30.899037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1633ea35eda34ffe884140eaab1ae909'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:31.085112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1290/2579] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:30 2022] DELETE /identity/v3/users/1633ea35eda34ffe884140eaab1ae909 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:31.574642 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d86f8-d81f-4741-8aaf-bd4196f5086a service placement] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:31.574642 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d86f8-d81f-4741-8aaf-bd4196f5086a service placement] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:31.602993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:31.647487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:31.648284 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:31.648568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:31.648852 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] PATH_INFO: `/v3/projects/92e9b7f9904f437f91148da9fc521326` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:31.650136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e9b7f9904f437f91148da9fc521326)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:31.652258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:31.707455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:31.709152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:31.733074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d66f08-402c-472a-b8a3-81e11afae944 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:31.738193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:31.738852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:31.744447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:31.744447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] PATH_INFO: `/v3/users/89bd12a289484b9594c29166e26c5ef5` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:31.744447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89bd12a289484b9594c29166e26c5ef5)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:31.744447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:31.792047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e9b7f9904f437f91148da9fc521326', 'request_id': 'req-3f14fe25-a3ce-4c71-82d1-eecb92515740'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:31.798613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e9b7f9904f437f91148da9fc521326', 'request_id': 'req-3f14fe25-a3ce-4c71-82d1-eecb92515740'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:31.898416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e9b7f9904f437f91148da9fc521326', 'request_id': 'req-3f14fe25-a3ce-4c71-82d1-eecb92515740'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:31.911651 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] The token cache is being invalidate because project 92e9b7f9904f437f91148da9fc521326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:31.911976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:31.912652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f14fe25-a3ce-4c71-82d1-eecb92515740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:31.914253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1290/2580] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:31 2022] DELETE /identity/v3/projects/92e9b7f9904f437f91148da9fc521326 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:31.920484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9414a92-0b39-4190-b4f3-76c24425b4d9 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:31.920918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9414a92-0b39-4190-b4f3-76c24425b4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:31.921211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9414a92-0b39-4190-b4f3-76c24425b4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:31.943850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a9414a92-0b39-4190-b4f3-76c24425b4d9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:31.945088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a9414a92-0b39-4190-b4f3-76c24425b4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:32.005975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89bd12a289484b9594c29166e26c5ef5'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:32.006761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89bd12a289484b9594c29166e26c5ef5'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:32.007541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89bd12a289484b9594c29166e26c5ef5'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:32.018311 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89bd12a289484b9594c29166e26c5ef5'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:32.034978 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1291/2581] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:32.038172 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0fa790df-3cb2-4122-808b-7f2819677830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89bd12a289484b9594c29166e26c5ef5'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:32.044824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f81db2-ce95-4a01-9afa-a987aa80f832 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:32.045105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f81db2-ce95-4a01-9afa-a987aa80f832 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:32.045347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f81db2-ce95-4a01-9afa-a987aa80f832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:32.067343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f5f81db2-ce95-4a01-9afa-a987aa80f832 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:32.068781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f5f81db2-ce95-4a01-9afa-a987aa80f832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:32.077903 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1292/2582] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:32.083034 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f81db2-ce95-4a01-9afa-a987aa80f832 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:32.093224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f81db2-ce95-4a01-9afa-a987aa80f832 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:32.098421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0032805-a4d1-4579-8652-6b0e1d7ec05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:32.099517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0032805-a4d1-4579-8652-6b0e1d7ec05c admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:32.099517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0032805-a4d1-4579-8652-6b0e1d7ec05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:32.099759 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0032805-a4d1-4579-8652-6b0e1d7ec05c admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:32.100350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0032805-a4d1-4579-8652-6b0e1d7ec05c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:37:32.100666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0032805-a4d1-4579-8652-6b0e1d7ec05c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:32.102384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0032805-a4d1-4579-8652-6b0e1d7ec05c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:32.120261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1293/2583] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:37:32.125255 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1291/2584] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:31 2022] DELETE /identity/v3/users/89bd12a289484b9594c29166e26c5ef5 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:32.134803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0032805-a4d1-4579-8652-6b0e1d7ec05c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:32.135023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c22cb-6aec-4b3d-b0d2-ad358e533f45 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:32.135023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c22cb-6aec-4b3d-b0d2-ad358e533f45 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:32.135023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c22cb-6aec-4b3d-b0d2-ad358e533f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:32.142680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0032805-a4d1-4579-8652-6b0e1d7ec05c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:32.158970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-2b5c22cb-6aec-4b3d-b0d2-ad358e533f45 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:32.160370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2b5c22cb-6aec-4b3d-b0d2-ad358e533f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:32.171767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:32.194833 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:32.229295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abda84-60bd-41e7-a906-bcdbfc1c5189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:32.230330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abda84-60bd-41e7-a906-bcdbfc1c5189 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:32.230824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abda84-60bd-41e7-a906-bcdbfc1c5189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:32.231320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abda84-60bd-41e7-a906-bcdbfc1c5189 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:32.232529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abda84-60bd-41e7-a906-bcdbfc1c5189 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:32.235795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abda84-60bd-41e7-a906-bcdbfc1c5189 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:32.237715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1292/2585] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:32.250512 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c22cb-6aec-4b3d-b0d2-ad358e533f45 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:32.254413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2b5c22cb-6aec-4b3d-b0d2-ad358e533f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:32.355663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c22cb-6aec-4b3d-b0d2-ad358e533f45 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:32.358929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1294/2586] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:32 2022] POST /identity/v3/projects => generated 375 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:32.364544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abda84-60bd-41e7-a906-bcdbfc1c5189 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:32.428996 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841e126-86fe-4013-8c48-7d732d3caf60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:32.430023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841e126-86fe-4013-8c48-7d732d3caf60 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:32.430317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841e126-86fe-4013-8c48-7d732d3caf60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:32.430597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841e126-86fe-4013-8c48-7d732d3caf60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:32.444840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abda84-60bd-41e7-a906-bcdbfc1c5189 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:32.460799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac34f5-7e82-4f0b-bf26-b09ba16ea408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:32.461718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac34f5-7e82-4f0b-bf26-b09ba16ea408 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:32.462164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac34f5-7e82-4f0b-bf26-b09ba16ea408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:32.463185 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaac34f5-7e82-4f0b-bf26-b09ba16ea408 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:32.464041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac34f5-7e82-4f0b-bf26-b09ba16ea408 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:32.472074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaac34f5-7e82-4f0b-bf26-b09ba16ea408 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:32.507578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841e126-86fe-4013-8c48-7d732d3caf60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:32.509028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841e126-86fe-4013-8c48-7d732d3caf60 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:32.520557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1293/2587] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:37:32.589583 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1295/2588] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:32 2022] POST /identity/v3/users => generated 288 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:32.594938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841e126-86fe-4013-8c48-7d732d3caf60 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:32.607302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841e126-86fe-4013-8c48-7d732d3caf60 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:32.613980 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a5101-ed81-40fe-be48-b287184dbc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:32.614646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a5101-ed81-40fe-be48-b287184dbc4d admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:32.614924 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a5101-ed81-40fe-be48-b287184dbc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:32.615206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a5101-ed81-40fe-be48-b287184dbc4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:32.615956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a5101-ed81-40fe-be48-b287184dbc4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:32.617622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a5101-ed81-40fe-be48-b287184dbc4d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:32.629861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1294/2589] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:32.635836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac34f5-7e82-4f0b-bf26-b09ba16ea408 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:32.651910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaac34f5-7e82-4f0b-bf26-b09ba16ea408 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:32.657600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9c98a-981c-4795-9eea-591b47a71434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:32.658264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9c98a-981c-4795-9eea-591b47a71434 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:32.661152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9c98a-981c-4795-9eea-591b47a71434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:32.661152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9c98a-981c-4795-9eea-591b47a71434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:32.661152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9c98a-981c-4795-9eea-591b47a71434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:32.661576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9c98a-981c-4795-9eea-591b47a71434 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:32.674449 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1296/2590] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:32.679066 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a5101-ed81-40fe-be48-b287184dbc4d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:32.690767 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a5101-ed81-40fe-be48-b287184dbc4d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:32.696356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922ee12-66df-40f3-ab4d-c84782d1fc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:32.697534 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922ee12-66df-40f3-ab4d-c84782d1fc09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:32.697815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922ee12-66df-40f3-ab4d-c84782d1fc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:32.698098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922ee12-66df-40f3-ab4d-c84782d1fc09 admin admin] PATH_INFO: `/v3/projects/b80d83d2a18d4830a9a9ed65b80b8f08/users/2645177d72c64f70a735d04a5c5529fc/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:32.710141 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:32.720214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922ee12-66df-40f3-ab4d-c84782d1fc09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80d83d2a18d4830a9a9ed65b80b8f08, user_id=2645177d72c64f70a735d04a5c5529fc, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:32.726186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922ee12-66df-40f3-ab4d-c84782d1fc09 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:32.862859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1295/2591] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:32 2022] PUT /identity/v3/projects/b80d83d2a18d4830a9a9ed65b80b8f08/users/2645177d72c64f70a735d04a5c5529fc/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:32.873958 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74e98-690f-4d10-bcf3-d059dc718fa0 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:32.874773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74e98-690f-4d10-bcf3-d059dc718fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:32.874773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def74e98-690f-4d10-bcf3-d059dc718fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:32.919183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-def74e98-690f-4d10-bcf3-d059dc718fa0 None None] MFA Rules not processed for user `2645177d72c64f70a735d04a5c5529fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:32.920858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-def74e98-690f-4d10-bcf3-d059dc718fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:33.011711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1297/2592] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:33.024659 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922ee12-66df-40f3-ab4d-c84782d1fc09 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:33.027165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f922ee12-66df-40f3-ab4d-c84782d1fc09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:33.082420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def74e98-690f-4d10-bcf3-d059dc718fa0 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:33.085256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-def74e98-690f-4d10-bcf3-d059dc718fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:33.120221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922ee12-66df-40f3-ab4d-c84782d1fc09 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:33.127420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc9310-e9ac-44f8-885a-7c423b429b6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:33.129115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc9310-e9ac-44f8-885a-7c423b429b6f service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:33.129474 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc9310-e9ac-44f8-885a-7c423b429b6f service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:33.129808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc9310-e9ac-44f8-885a-7c423b429b6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:33.141687 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc9310-e9ac-44f8-885a-7c423b429b6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:33.143395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc9310-e9ac-44f8-885a-7c423b429b6f service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:33.147728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:33.154238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1296/2593] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:33.185696 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def74e98-690f-4d10-bcf3-d059dc718fa0 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:33.203451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:33.203451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:33.203451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:33.203451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] PATH_INFO: `/v3/projects/069a6d65c8f543c489870965355a8214` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:33.204357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=069a6d65c8f543c489870965355a8214)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:33.204357 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:33.326590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069a6d65c8f543c489870965355a8214', 'request_id': 'req-0fde6b4b-568d-4d0a-a363-dca6e243f876'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.334429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069a6d65c8f543c489870965355a8214', 'request_id': 'req-0fde6b4b-568d-4d0a-a363-dca6e243f876'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.407350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069a6d65c8f543c489870965355a8214', 'request_id': 'req-0fde6b4b-568d-4d0a-a363-dca6e243f876'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.429153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] The token cache is being invalidate because project 069a6d65c8f543c489870965355a8214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:33.429153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.429153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.429153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1298/2594] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:33 2022] DELETE /identity/v3/projects/069a6d65c8f543c489870965355a8214 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:33.429153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:33.431789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc9310-e9ac-44f8-885a-7c423b429b6f service nova] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:33.433119 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:33.440925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc9310-e9ac-44f8-885a-7c423b429b6f service nova] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:33.512730 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde6b4b-568d-4d0a-a363-dca6e243f876 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:33.516698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:33.517553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:33.517939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:33.518315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] PATH_INFO: `/v3/users/b8bfd6e0b1c440169ea0698f9c3e3dd0` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:33.519839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8bfd6e0b1c440169ea0698f9c3e3dd0)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:33.521577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:33.522238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:33.522435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:33.522690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:33.522972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] PATH_INFO: `/v3/users/2645177d72c64f70a735d04a5c5529fc` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:33.525621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2645177d72c64f70a735d04a5c5529fc)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:33.529442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:33.782215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2645177d72c64f70a735d04a5c5529fc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.796352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8bfd6e0b1c440169ea0698f9c3e3dd0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.797576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8bfd6e0b1c440169ea0698f9c3e3dd0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.798586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8bfd6e0b1c440169ea0698f9c3e3dd0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.810872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8bfd6e0b1c440169ea0698f9c3e3dd0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.826030 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8bfd6e0b1c440169ea0698f9c3e3dd0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.828016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:33.864084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2645177d72c64f70a735d04a5c5529fc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.864944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2645177d72c64f70a735d04a5c5529fc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.872871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:33.883216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2645177d72c64f70a735d04a5c5529fc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.884563 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-485312d3-8bc7-4bd8-9e04-5ec2ffe474fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2645177d72c64f70a735d04a5c5529fc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:33.899301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1299/2595] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:33 2022] DELETE /identity/v3/users/2645177d72c64f70a735d04a5c5529fc => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:33.906459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbb4bc-a3b4-4f3f-bcf8-37aee465ba99 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:33.906970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbb4bc-a3b4-4f3f-bcf8-37aee465ba99 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:33.907436 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbb4bc-a3b4-4f3f-bcf8-37aee465ba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:33.932767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-5ddbb4bc-a3b4-4f3f-bcf8-37aee465ba99 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:33.934153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5ddbb4bc-a3b4-4f3f-bcf8-37aee465ba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:33.966269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1297/2596] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:33 2022] DELETE /identity/v3/users/b8bfd6e0b1c440169ea0698f9c3e3dd0 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:34.044736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1300/2597] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:34.054248 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:34.056040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:34.142208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9d3c0-8b1c-4b44-b6da-39caea0031ac admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:34.145942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fdb65-1da7-455a-a56e-0136b3289e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:34.146567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fdb65-1da7-455a-a56e-0136b3289e36 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:34.146843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fdb65-1da7-455a-a56e-0136b3289e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:34.147114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fdb65-1da7-455a-a56e-0136b3289e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:34.227615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fdb65-1da7-455a-a56e-0136b3289e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:34.229764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fdb65-1da7-455a-a56e-0136b3289e36 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:34.249569 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1298/2598] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:34.554120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbb4bc-a3b4-4f3f-bcf8-37aee465ba99 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:34.557535 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5ddbb4bc-a3b4-4f3f-bcf8-37aee465ba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:34.581088 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbb4bc-a3b4-4f3f-bcf8-37aee465ba99 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:34.588783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:34.590047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:34.590475 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:34.591055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] PATH_INFO: `/v3/projects/dc47ca5f143044a0b6ff075f57efac61` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:34.593260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc47ca5f143044a0b6ff075f57efac61)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:34.597733 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:34.877159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc47ca5f143044a0b6ff075f57efac61', 'request_id': 'req-5f586db2-df6d-4de1-a32a-02bf00806130'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:34.886736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc47ca5f143044a0b6ff075f57efac61', 'request_id': 'req-5f586db2-df6d-4de1-a32a-02bf00806130'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:34.904470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fdb65-1da7-455a-a56e-0136b3289e36 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:34.922405 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-048fdb65-1da7-455a-a56e-0136b3289e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:34.931949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fdb65-1da7-455a-a56e-0136b3289e36 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.014359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc47ca5f143044a0b6ff075f57efac61', 'request_id': 'req-5f586db2-df6d-4de1-a32a-02bf00806130'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:35.019780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.020451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.020737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.021187 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] PATH_INFO: `/v3/projects/b80d83d2a18d4830a9a9ed65b80b8f08` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.022724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b80d83d2a18d4830a9a9ed65b80b8f08)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.025757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.033614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] The token cache is being invalidate because project dc47ca5f143044a0b6ff075f57efac61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:35.034094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:35.035243 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-5f586db2-df6d-4de1-a32a-02bf00806130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:35.037896 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1301/2599] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:34 2022] DELETE /identity/v3/projects/dc47ca5f143044a0b6ff075f57efac61 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:35.045780 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf870f1-6209-4415-b118-f5067b463f54 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.046269 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf870f1-6209-4415-b118-f5067b463f54 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.046625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf870f1-6209-4415-b118-f5067b463f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.074060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-daf870f1-6209-4415-b118-f5067b463f54 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:35.075516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-daf870f1-6209-4415-b118-f5067b463f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:35.084841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1302/2600] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:35.102818 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d29b06-225b-44af-a16a-5cd78bc6305a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.102818 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d29b06-225b-44af-a16a-5cd78bc6305a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.102818 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d29b06-225b-44af-a16a-5cd78bc6305a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.124169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-28d29b06-225b-44af-a16a-5cd78bc6305a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:35.125497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-28d29b06-225b-44af-a16a-5cd78bc6305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:35.143454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1303/2601] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:35.149568 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d29b06-225b-44af-a16a-5cd78bc6305a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.169076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d29b06-225b-44af-a16a-5cd78bc6305a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.176199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7db49-5071-444a-b9b1-b79c30a3392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.176834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7db49-5071-444a-b9b1-b79c30a3392d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.177110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7db49-5071-444a-b9b1-b79c30a3392d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.177420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7db49-5071-444a-b9b1-b79c30a3392d admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.178178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7db49-5071-444a-b9b1-b79c30a3392d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:37:35.178735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7db49-5071-444a-b9b1-b79c30a3392d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.180217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7db49-5071-444a-b9b1-b79c30a3392d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.198281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1304/2602] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:37:35.199148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:INSERT=1,keystone:DELETE=4,keystone:SELECT=95 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:35.204321 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7db49-5071-444a-b9b1-b79c30a3392d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.206134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b80d83d2a18d4830a9a9ed65b80b8f08', 'request_id': 'req-22c9af66-3712-460e-8360-7b3390a9e29c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:35.217564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7db49-5071-444a-b9b1-b79c30a3392d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.224165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b80d83d2a18d4830a9a9ed65b80b8f08', 'request_id': 'req-22c9af66-3712-460e-8360-7b3390a9e29c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:35.269293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:35.300105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee532f89-7393-47e8-8c51-003338eda75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.300797 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee532f89-7393-47e8-8c51-003338eda75a admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.301113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee532f89-7393-47e8-8c51-003338eda75a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.301454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee532f89-7393-47e8-8c51-003338eda75a admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.302331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee532f89-7393-47e8-8c51-003338eda75a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.304640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee532f89-7393-47e8-8c51-003338eda75a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.317724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b80d83d2a18d4830a9a9ed65b80b8f08', 'request_id': 'req-22c9af66-3712-460e-8360-7b3390a9e29c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:35.333879 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] The token cache is being invalidate because project b80d83d2a18d4830a9a9ed65b80b8f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:35.334231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:35.335009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-22c9af66-3712-460e-8360-7b3390a9e29c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:35.336713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1299/2603] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:34 2022] DELETE /identity/v3/projects/b80d83d2a18d4830a9a9ed65b80b8f08 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:35.343490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e755-f762-4fdf-b5e0-aa8bf325615e None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.344070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e755-f762-4fdf-b5e0-aa8bf325615e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.344423 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5e755-f762-4fdf-b5e0-aa8bf325615e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.370590 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6bf5e755-f762-4fdf-b5e0-aa8bf325615e None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:35.371966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6bf5e755-f762-4fdf-b5e0-aa8bf325615e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:35.381240 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1300/2604] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:35.392063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27126f34-4f66-45db-944e-99d5c66c211a None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.392356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27126f34-4f66-45db-944e-99d5c66c211a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.392641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27126f34-4f66-45db-944e-99d5c66c211a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.416537 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-27126f34-4f66-45db-944e-99d5c66c211a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:35.417718 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-27126f34-4f66-45db-944e-99d5c66c211a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:35.426695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1301/2605] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:35.433088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27126f34-4f66-45db-944e-99d5c66c211a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.445602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27126f34-4f66-45db-944e-99d5c66c211a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.452281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2704a-46c3-4632-9267-edce74813673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.457938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2704a-46c3-4632-9267-edce74813673 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.457938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2704a-46c3-4632-9267-edce74813673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.457938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2704a-46c3-4632-9267-edce74813673 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.457938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2704a-46c3-4632-9267-edce74813673 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:37:35.457938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2704a-46c3-4632-9267-edce74813673 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.457938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2704a-46c3-4632-9267-edce74813673 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.473412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1302/2606] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:37:35.478470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2704a-46c3-4632-9267-edce74813673 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.490073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2704a-46c3-4632-9267-edce74813673 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.495710 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1305/2607] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:35 2022] POST /identity/v3/projects => generated 407 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:35.495935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc88bb9-ba35-492e-8715-de5b17983120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.496416 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc88bb9-ba35-492e-8715-de5b17983120 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.496785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc88bb9-ba35-492e-8715-de5b17983120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.497062 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc88bb9-ba35-492e-8715-de5b17983120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.497730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc88bb9-ba35-492e-8715-de5b17983120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.499997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc88bb9-ba35-492e-8715-de5b17983120 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.502499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee532f89-7393-47e8-8c51-003338eda75a admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.506093 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ee532f89-7393-47e8-8c51-003338eda75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:35.603409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee532f89-7393-47e8-8c51-003338eda75a admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.609098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2e647-de0a-422c-8fd5-3cd51b1e45ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.609782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2e647-de0a-422c-8fd5-3cd51b1e45ca admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.610137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2e647-de0a-422c-8fd5-3cd51b1e45ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.610564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2e647-de0a-422c-8fd5-3cd51b1e45ca admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.611473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2e647-de0a-422c-8fd5-3cd51b1e45ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.613762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2e647-de0a-422c-8fd5-3cd51b1e45ca admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.622081 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1303/2608] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:35 2022] POST /identity/v3/projects => generated 391 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:35.623320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc88bb9-ba35-492e-8715-de5b17983120 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.715768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1306/2609] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:35 2022] POST /identity/v3/users => generated 304 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:35.721140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc88bb9-ba35-492e-8715-de5b17983120 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.721389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2e647-de0a-422c-8fd5-3cd51b1e45ca admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.727360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1a1d2-cca5-499c-84d7-310a8ecdbcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.728189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1a1d2-cca5-499c-84d7-310a8ecdbcab admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.728487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1a1d2-cca5-499c-84d7-310a8ecdbcab admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.728769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1a1d2-cca5-499c-84d7-310a8ecdbcab admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.729458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1a1d2-cca5-499c-84d7-310a8ecdbcab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.731504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1a1d2-cca5-499c-84d7-310a8ecdbcab admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.736376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2e647-de0a-422c-8fd5-3cd51b1e45ca admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.744456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16f285-fc5a-4b53-aed6-6d8a3d6d148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.745207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16f285-fc5a-4b53-aed6-6d8a3d6d148b admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.749812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16f285-fc5a-4b53-aed6-6d8a3d6d148b admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.749812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f16f285-fc5a-4b53-aed6-6d8a3d6d148b admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.749812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16f285-fc5a-4b53-aed6-6d8a3d6d148b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.749812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f16f285-fc5a-4b53-aed6-6d8a3d6d148b admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.762272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1307/2610] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:35.768075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16f285-fc5a-4b53-aed6-6d8a3d6d148b admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.785018 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f16f285-fc5a-4b53-aed6-6d8a3d6d148b admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.790594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b3792-7c6e-413a-a316-d7792b01c1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.791427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b3792-7c6e-413a-a316-d7792b01c1bf admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.791832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b3792-7c6e-413a-a316-d7792b01c1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.792165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7b3792-7c6e-413a-a316-d7792b01c1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.792899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b3792-7c6e-413a-a316-d7792b01c1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.794679 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7b3792-7c6e-413a-a316-d7792b01c1bf admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.808564 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1308/2611] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:37:35.813997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b3792-7c6e-413a-a316-d7792b01c1bf admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.826304 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7b3792-7c6e-413a-a316-d7792b01c1bf admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.832500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b32ed-2a72-4ff6-8c4f-479fc4662372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.833453 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b32ed-2a72-4ff6-8c4f-479fc4662372 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.834355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b32ed-2a72-4ff6-8c4f-479fc4662372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.834355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b32ed-2a72-4ff6-8c4f-479fc4662372 admin admin] PATH_INFO: `/v3/projects/5b20c1a0b392452b8ca1a6813ea18cbb/users/b2ef96e484334796840f8c8ff1aeb887/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.854637 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b32ed-2a72-4ff6-8c4f-479fc4662372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b20c1a0b392452b8ca1a6813ea18cbb, user_id=b2ef96e484334796840f8c8ff1aeb887, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.858409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b32ed-2a72-4ff6-8c4f-479fc4662372 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.865533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1304/2612] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:35 2022] POST /identity/v3/users => generated 296 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:35.867053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1a1d2-cca5-499c-84d7-310a8ecdbcab admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.882332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1a1d2-cca5-499c-84d7-310a8ecdbcab admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.889247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42810f41-05d9-499c-88ce-326b9e5dc9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.890006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42810f41-05d9-499c-88ce-326b9e5dc9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.890389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42810f41-05d9-499c-88ce-326b9e5dc9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.890762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42810f41-05d9-499c-88ce-326b9e5dc9d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.891644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42810f41-05d9-499c-88ce-326b9e5dc9d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.893698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42810f41-05d9-499c-88ce-326b9e5dc9d6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.911403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1305/2613] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:35.917135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42810f41-05d9-499c-88ce-326b9e5dc9d6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.928770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42810f41-05d9-499c-88ce-326b9e5dc9d6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.934810 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac6fd6-64de-4896-b47b-5115a629b802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:35.935441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac6fd6-64de-4896-b47b-5115a629b802 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.935764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac6fd6-64de-4896-b47b-5115a629b802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.936043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac6fd6-64de-4896-b47b-5115a629b802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:35.936713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac6fd6-64de-4896-b47b-5115a629b802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:35.938569 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac6fd6-64de-4896-b47b-5115a629b802 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:35.952403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1306/2614] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:37:35.957338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac6fd6-64de-4896-b47b-5115a629b802 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:35.958809 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1309/2615] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:35 2022] PUT /identity/v3/projects/5b20c1a0b392452b8ca1a6813ea18cbb/users/b2ef96e484334796840f8c8ff1aeb887/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:35.971000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f940d-5a1a-431f-9d52-f1c100318035 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:35.971254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac6fd6-64de-4896-b47b-5115a629b802 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:35.971414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f940d-5a1a-431f-9d52-f1c100318035 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:35.972027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f940d-5a1a-431f-9d52-f1c100318035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.008876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-379f940d-5a1a-431f-9d52-f1c100318035 None None] MFA Rules not processed for user `b2ef96e484334796840f8c8ff1aeb887`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:36.010214 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-379f940d-5a1a-431f-9d52-f1c100318035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:36.043482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:36.054628 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa2211-79d7-4ae2-a78c-f24f37efe4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:36.055343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa2211-79d7-4ae2-a78c-f24f37efe4f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.055623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa2211-79d7-4ae2-a78c-f24f37efe4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.055964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa2211-79d7-4ae2-a78c-f24f37efe4f3 admin admin] PATH_INFO: `/v3/projects/5857e8abbb8c466da8997a99fbda443c/users/8af61482764b4d09bfe600e35661fef2/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.075744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa2211-79d7-4ae2-a78c-f24f37efe4f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5857e8abbb8c466da8997a99fbda443c, user_id=8af61482764b4d09bfe600e35661fef2, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:36.079282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa2211-79d7-4ae2-a78c-f24f37efe4f3 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:36.104104 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:36.116882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1310/2616] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:35 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:36.122382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f940d-5a1a-431f-9d52-f1c100318035 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:36.135466 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f940d-5a1a-431f-9d52-f1c100318035 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:36.140844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e8230-fd04-4607-a803-4b61a4f3093e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:36.141487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e8230-fd04-4607-a803-4b61a4f3093e admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.141766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e8230-fd04-4607-a803-4b61a4f3093e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.142047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e8230-fd04-4607-a803-4b61a4f3093e admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.142716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e8230-fd04-4607-a803-4b61a4f3093e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:36.144910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e8230-fd04-4607-a803-4b61a4f3093e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:36.219595 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1307/2617] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:35 2022] PUT /identity/v3/projects/5857e8abbb8c466da8997a99fbda443c/users/8af61482764b4d09bfe600e35661fef2/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:36.232432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b856201-1d6a-49c5-bbed-fe266ee3aea3 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.233063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b856201-1d6a-49c5-bbed-fe266ee3aea3 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.233366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b856201-1d6a-49c5-bbed-fe266ee3aea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.250973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1311/2618] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:36 2022] POST /identity/v3/projects => generated 409 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:36.255692 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e8230-fd04-4607-a803-4b61a4f3093e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:36.273594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6b856201-1d6a-49c5-bbed-fe266ee3aea3 None None] MFA Rules not processed for user `8af61482764b4d09bfe600e35661fef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:36.276086 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6b856201-1d6a-49c5-bbed-fe266ee3aea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:36.370782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e8230-fd04-4607-a803-4b61a4f3093e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:36.378942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86594263-d302-4bb7-9272-2c34d86996e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:36.380497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86594263-d302-4bb7-9272-2c34d86996e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.381137 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86594263-d302-4bb7-9272-2c34d86996e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.381645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86594263-d302-4bb7-9272-2c34d86996e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.382913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86594263-d302-4bb7-9272-2c34d86996e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:36.387603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86594263-d302-4bb7-9272-2c34d86996e2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:36.397009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1308/2619] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:36.404902 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b856201-1d6a-49c5-bbed-fe266ee3aea3 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:36.423327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b856201-1d6a-49c5-bbed-fe266ee3aea3 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:36.430411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a70fb3-0abe-4718-8049-9c6be9546978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:36.431467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a70fb3-0abe-4718-8049-9c6be9546978 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.431990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a70fb3-0abe-4718-8049-9c6be9546978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.432429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a70fb3-0abe-4718-8049-9c6be9546978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.433215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a70fb3-0abe-4718-8049-9c6be9546978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:36.436312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a70fb3-0abe-4718-8049-9c6be9546978 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:36.553805 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1312/2620] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:36 2022] POST /identity/v3/users => generated 311 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:37:36.559821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86594263-d302-4bb7-9272-2c34d86996e2 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:36.572490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1309/2621] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:36 2022] POST /identity/v3/projects => generated 393 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:36.574169 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86594263-d302-4bb7-9272-2c34d86996e2 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:36.578140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a70fb3-0abe-4718-8049-9c6be9546978 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:36.670663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9aa943-3deb-45a8-892f-8c4e9ec9f1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:36.672180 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9aa943-3deb-45a8-892f-8c4e9ec9f1cc admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.672459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9aa943-3deb-45a8-892f-8c4e9ec9f1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.672738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9aa943-3deb-45a8-892f-8c4e9ec9f1cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.673487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9aa943-3deb-45a8-892f-8c4e9ec9f1cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:36.675211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9aa943-3deb-45a8-892f-8c4e9ec9f1cc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:36.685526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a70fb3-0abe-4718-8049-9c6be9546978 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:36.689370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3de8ca-9f31-4c25-b418-0bfd3d495c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:36.690035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3de8ca-9f31-4c25-b418-0bfd3d495c14 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.690291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3de8ca-9f31-4c25-b418-0bfd3d495c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.690592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3de8ca-9f31-4c25-b418-0bfd3d495c14 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.691516 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3de8ca-9f31-4c25-b418-0bfd3d495c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:36.694155 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1313/2622] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:37:36.694554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3de8ca-9f31-4c25-b418-0bfd3d495c14 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:36.699979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9aa943-3deb-45a8-892f-8c4e9ec9f1cc admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:36.720336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9aa943-3deb-45a8-892f-8c4e9ec9f1cc admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:36.732795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437711-e81f-40e4-98c4-404252772ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:36.733492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437711-e81f-40e4-98c4-404252772ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.733778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437711-e81f-40e4-98c4-404252772ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.734057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db437711-e81f-40e4-98c4-404252772ba9 admin admin] PATH_INFO: `/v3/projects/28bde6da6d634c37925bd7cddd0c2720/users/efcb8153b8a2426a9acbae87fe774e78/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.753729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db437711-e81f-40e4-98c4-404252772ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bde6da6d634c37925bd7cddd0c2720, user_id=efcb8153b8a2426a9acbae87fe774e78, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:36.756477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db437711-e81f-40e4-98c4-404252772ba9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:36.804824 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1310/2623] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:36 2022] POST /identity/v3/users => generated 303 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:36.811035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3de8ca-9f31-4c25-b418-0bfd3d495c14 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:36.832679 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3de8ca-9f31-4c25-b418-0bfd3d495c14 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:36.843307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1314/2624] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:36 2022] PUT /identity/v3/projects/28bde6da6d634c37925bd7cddd0c2720/users/efcb8153b8a2426a9acbae87fe774e78/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:36.848918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437711-e81f-40e4-98c4-404252772ba9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:36.883827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:36.922810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:36.933219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd74dd-8509-4f6f-bfc5-9567be195581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:36.934260 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd74dd-8509-4f6f-bfc5-9567be195581 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.934756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd74dd-8509-4f6f-bfc5-9567be195581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.935237 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd74dd-8509-4f6f-bfc5-9567be195581 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.936566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd74dd-8509-4f6f-bfc5-9567be195581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:36.938992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd74dd-8509-4f6f-bfc5-9567be195581 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:36.952135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1311/2625] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:37:36.958350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd74dd-8509-4f6f-bfc5-9567be195581 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:36.971955 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd74dd-8509-4f6f-bfc5-9567be195581 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:36.974511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db437711-e81f-40e4-98c4-404252772ba9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:36.981936 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7ae82-7e17-4c24-8c90-518cd3019903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:36.982258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7ae82-7e17-4c24-8c90-518cd3019903 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.982428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7ae82-7e17-4c24-8c90-518cd3019903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.982596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7ae82-7e17-4c24-8c90-518cd3019903 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:36.983253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7ae82-7e17-4c24-8c90-518cd3019903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:36.985135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c7ae82-7e17-4c24-8c90-518cd3019903 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:36.988183 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487ce44-bada-4864-9aef-c6de009eef72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:36.989014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487ce44-bada-4864-9aef-c6de009eef72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:36.989317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487ce44-bada-4864-9aef-c6de009eef72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:36.989831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b487ce44-bada-4864-9aef-c6de009eef72 admin admin] PATH_INFO: `/v3/projects/a9a7145af1b3449883e9403cb77cdb5f/users/c12e9614eb9046919312e97c25fb970d/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:37.005451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1315/2626] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:37:37.010431 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7ae82-7e17-4c24-8c90-518cd3019903 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:37.012119 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487ce44-bada-4864-9aef-c6de009eef72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a7145af1b3449883e9403cb77cdb5f, user_id=c12e9614eb9046919312e97c25fb970d, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:37.015266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b487ce44-bada-4864-9aef-c6de009eef72 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:37.023606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7ae82-7e17-4c24-8c90-518cd3019903 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:37.030211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825bc29-cad9-4877-a35b-29651d943a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:37.031112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825bc29-cad9-4877-a35b-29651d943a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:37.031420 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825bc29-cad9-4877-a35b-29651d943a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:37.031749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825bc29-cad9-4877-a35b-29651d943a70 admin admin] PATH_INFO: `/v3/projects/28bde6da6d634c37925bd7cddd0c2720/users/efcb8153b8a2426a9acbae87fe774e78/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:37.034247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825bc29-cad9-4877-a35b-29651d943a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28bde6da6d634c37925bd7cddd0c2720, user_id=efcb8153b8a2426a9acbae87fe774e78, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:37.037429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825bc29-cad9-4877-a35b-29651d943a70 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:37.052919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1316/2627] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:37 2022] PUT /identity/v3/projects/28bde6da6d634c37925bd7cddd0c2720/users/efcb8153b8a2426a9acbae87fe774e78/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:37.066419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb477998-34d9-420a-ba88-8f7bb138bc19 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:37.066690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb477998-34d9-420a-ba88-8f7bb138bc19 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:37.066879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb477998-34d9-420a-ba88-8f7bb138bc19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:37.105465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-eb477998-34d9-420a-ba88-8f7bb138bc19 None None] MFA Rules not processed for user `efcb8153b8a2426a9acbae87fe774e78`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:37.106720 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-eb477998-34d9-420a-ba88-8f7bb138bc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:37.128643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1312/2628] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:36 2022] PUT /identity/v3/projects/a9a7145af1b3449883e9403cb77cdb5f/users/c12e9614eb9046919312e97c25fb970d/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:37.135350 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487ce44-bada-4864-9aef-c6de009eef72 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:37.242252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1317/2629] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:37 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:37.256424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb477998-34d9-420a-ba88-8f7bb138bc19 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:37.259449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b487ce44-bada-4864-9aef-c6de009eef72 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:37.264606 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-eb477998-34d9-420a-ba88-8f7bb138bc19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:37.271095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e790c9-d0af-4f31-82fc-0efb12054645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:37.272325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e790c9-d0af-4f31-82fc-0efb12054645 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:37.272805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e790c9-d0af-4f31-82fc-0efb12054645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:37.273279 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e790c9-d0af-4f31-82fc-0efb12054645 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:37.274386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e790c9-d0af-4f31-82fc-0efb12054645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:37.286269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e790c9-d0af-4f31-82fc-0efb12054645 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:37.307725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1313/2630] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:37 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:37.314622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e790c9-d0af-4f31-82fc-0efb12054645 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:37.341982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e790c9-d0af-4f31-82fc-0efb12054645 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:37.345298 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb43a7-5a25-4ce3-bfb3-f2ce03cea77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:37.346962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb43a7-5a25-4ce3-bfb3-f2ce03cea77a admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:37.347748 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb43a7-5a25-4ce3-bfb3-f2ce03cea77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:37.348359 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb43a7-5a25-4ce3-bfb3-f2ce03cea77a admin admin] PATH_INFO: `/v3/projects/a9a7145af1b3449883e9403cb77cdb5f/users/c12e9614eb9046919312e97c25fb970d/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:37.353893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb43a7-5a25-4ce3-bfb3-f2ce03cea77a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a7145af1b3449883e9403cb77cdb5f, user_id=c12e9614eb9046919312e97c25fb970d, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:37.359331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb43a7-5a25-4ce3-bfb3-f2ce03cea77a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:37.371750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb477998-34d9-420a-ba88-8f7bb138bc19 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:37.382769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1314/2631] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:37 2022] PUT /identity/v3/projects/a9a7145af1b3449883e9403cb77cdb5f/users/c12e9614eb9046919312e97c25fb970d/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:37.396820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad1f22-e36e-4fc6-b44f-871f39ca8d1a None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:37.397400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad1f22-e36e-4fc6-b44f-871f39ca8d1a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:37.397860 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad1f22-e36e-4fc6-b44f-871f39ca8d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:37.443360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-acad1f22-e36e-4fc6-b44f-871f39ca8d1a None None] MFA Rules not processed for user `c12e9614eb9046919312e97c25fb970d`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:37.444915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-acad1f22-e36e-4fc6-b44f-871f39ca8d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:37.476971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7033df-a1bc-4fc9-a0de-573f570674e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:37.478881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7033df-a1bc-4fc9-a0de-573f570674e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:37.478881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7033df-a1bc-4fc9-a0de-573f570674e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:37.478881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7033df-a1bc-4fc9-a0de-573f570674e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:37.564044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7033df-a1bc-4fc9-a0de-573f570674e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:37.565596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7033df-a1bc-4fc9-a0de-573f570674e8 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:37.579493 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1315/2632] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:37 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:37.589685 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1318/2633] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:37 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:37:37.598460 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad1f22-e36e-4fc6-b44f-871f39ca8d1a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:37.601113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-acad1f22-e36e-4fc6-b44f-871f39ca8d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:37.610165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:37.620170 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad1f22-e36e-4fc6-b44f-871f39ca8d1a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:37.625388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8703f90-8b2f-44bc-963e-37ff720731bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:37.626283 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8703f90-8b2f-44bc-963e-37ff720731bd service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:37.626648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8703f90-8b2f-44bc-963e-37ff720731bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:37.626974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8703f90-8b2f-44bc-963e-37ff720731bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:37.640101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8703f90-8b2f-44bc-963e-37ff720731bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:37.642014 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8703f90-8b2f-44bc-963e-37ff720731bd service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:37.655602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1316/2634] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:37 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:37.656957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:37.736220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7033df-a1bc-4fc9-a0de-573f570674e8 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:37.754975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7033df-a1bc-4fc9-a0de-573f570674e8 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:37.764900 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317be4bd-d699-42cc-a9bf-b71b945f616f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:37.766648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317be4bd-d699-42cc-a9bf-b71b945f616f service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:37.766648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317be4bd-d699-42cc-a9bf-b71b945f616f service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:37.766648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317be4bd-d699-42cc-a9bf-b71b945f616f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:37.864235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317be4bd-d699-42cc-a9bf-b71b945f616f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:37.865947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317be4bd-d699-42cc-a9bf-b71b945f616f service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:37.879877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1319/2635] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:38.097532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8703f90-8b2f-44bc-963e-37ff720731bd service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:38.110944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8703f90-8b2f-44bc-963e-37ff720731bd service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:38.115186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae3b2a-5b64-4a19-8c93-d99982277f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:38.115777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae3b2a-5b64-4a19-8c93-d99982277f80 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:38.116047 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae3b2a-5b64-4a19-8c93-d99982277f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:38.116486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae3b2a-5b64-4a19-8c93-d99982277f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:38.214371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae3b2a-5b64-4a19-8c93-d99982277f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:38.217685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae3b2a-5b64-4a19-8c93-d99982277f80 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:38.238862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1317/2636] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:38 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:46.570617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317be4bd-d699-42cc-a9bf-b71b945f616f service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:46.684064 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317be4bd-d699-42cc-a9bf-b71b945f616f service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:46.689533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:46.690268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:46.690592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:46.691017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] PATH_INFO: `/v3/users/8af61482764b4d09bfe600e35661fef2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:46.692404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8af61482764b4d09bfe600e35661fef2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:46.694556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:46.941729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8af61482764b4d09bfe600e35661fef2'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:46.949160 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:47.083665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8af61482764b4d09bfe600e35661fef2'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:47.084678 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8af61482764b4d09bfe600e35661fef2'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:47.103479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8af61482764b4d09bfe600e35661fef2'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:47.104544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8af61482764b4d09bfe600e35661fef2'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:47.119617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1320/2637] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:46 2022] DELETE /identity/v3/users/8af61482764b4d09bfe600e35661fef2 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:47.127012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29d34-daf3-40ff-94ae-346ce725688b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:47.127630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29d34-daf3-40ff-94ae-346ce725688b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:47.128073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee29d34-daf3-40ff-94ae-346ce725688b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:47.158597 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6ee29d34-daf3-40ff-94ae-346ce725688b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:47.160153 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6ee29d34-daf3-40ff-94ae-346ce725688b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:47.262521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1318/2638] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:47.278473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:47.289417 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:47.418531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f088532-765d-4087-b945-a6086f44effc admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:47.427051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5bab-dc1e-4240-8647-f73baca4d396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:47.427454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5bab-dc1e-4240-8647-f73baca4d396 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:47.428239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5bab-dc1e-4240-8647-f73baca4d396 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:47.428933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5bab-dc1e-4240-8647-f73baca4d396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:47.448132 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de5bab-dc1e-4240-8647-f73baca4d396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:47.450682 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de5bab-dc1e-4240-8647-f73baca4d396 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:47.471223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1321/2639] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:48.004347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29d34-daf3-40ff-94ae-346ce725688b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:48.006475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6ee29d34-daf3-40ff-94ae-346ce725688b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:48.028461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee29d34-daf3-40ff-94ae-346ce725688b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:48.037515 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:48.038688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:48.039210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:48.039776 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] PATH_INFO: `/v3/projects/5857e8abbb8c466da8997a99fbda443c` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:48.042060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5857e8abbb8c466da8997a99fbda443c)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:48.046658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:48.202281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5857e8abbb8c466da8997a99fbda443c', 'request_id': 'req-fae0f9d8-2565-4272-8b3c-1ff4d0316727'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:48.209999 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5857e8abbb8c466da8997a99fbda443c', 'request_id': 'req-fae0f9d8-2565-4272-8b3c-1ff4d0316727'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:48.351167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5857e8abbb8c466da8997a99fbda443c', 'request_id': 'req-fae0f9d8-2565-4272-8b3c-1ff4d0316727'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:48.363362 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] The token cache is being invalidate because project 5857e8abbb8c466da8997a99fbda443c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:48.363727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:48.364408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:48.367476 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1319/2640] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:48 2022] DELETE /identity/v3/projects/5857e8abbb8c466da8997a99fbda443c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:48.370904 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5bab-dc1e-4240-8647-f73baca4d396 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:48.383368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5bab-dc1e-4240-8647-f73baca4d396 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:48.451515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:48.452221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:48.452586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:48.452818 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] PATH_INFO: `/v3/users/c12e9614eb9046919312e97c25fb970d` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:48.454203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c12e9614eb9046919312e97c25fb970d)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:48.456344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:48.680166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c12e9614eb9046919312e97c25fb970d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:48.928551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c12e9614eb9046919312e97c25fb970d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:48.929301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c12e9614eb9046919312e97c25fb970d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:48.947472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:48.947472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c12e9614eb9046919312e97c25fb970d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:48.947472 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c12e9614eb9046919312e97c25fb970d'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:48.968479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1322/2641] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:48 2022] DELETE /identity/v3/users/c12e9614eb9046919312e97c25fb970d => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:49.660893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:49.663373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:49.685120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae0f9d8-2565-4272-8b3c-1ff4d0316727 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:49.713405 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:49.771716 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:49.772946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:49.773464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:49.773907 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] PATH_INFO: `/v3/projects/a9a7145af1b3449883e9403cb77cdb5f` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:49.776044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a7145af1b3449883e9403cb77cdb5f)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:49.780342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:49.803369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:49.809248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:49.834197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ea59c-ac5d-4cfe-93f2-ab8155fa5a76 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:49.843359 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:49.844946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:49.845459 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:49.846024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] PATH_INFO: `/v3/users/b2ef96e484334796840f8c8ff1aeb887` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:49.848987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ef96e484334796840f8c8ff1aeb887)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:49.853859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:49.969898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a7145af1b3449883e9403cb77cdb5f', 'request_id': 'req-f5a54b16-48ae-4b57-ac32-31702bcd46df'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:49.984122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a7145af1b3449883e9403cb77cdb5f', 'request_id': 'req-f5a54b16-48ae-4b57-ac32-31702bcd46df'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:50.097935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a7145af1b3449883e9403cb77cdb5f', 'request_id': 'req-f5a54b16-48ae-4b57-ac32-31702bcd46df'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:50.106027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ef96e484334796840f8c8ff1aeb887'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:50.119816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] The token cache is being invalidate because project a9a7145af1b3449883e9403cb77cdb5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:50.120324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:50.121558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-f5a54b16-48ae-4b57-ac32-31702bcd46df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:50.124566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1320/2642] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:49 2022] DELETE /identity/v3/projects/a9a7145af1b3449883e9403cb77cdb5f => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:50.132601 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cacc7-8b0f-41b8-9cfa-3e3d7c3bc114 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:50.133040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cacc7-8b0f-41b8-9cfa-3e3d7c3bc114 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:50.133366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cacc7-8b0f-41b8-9cfa-3e3d7c3bc114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:50.165054 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-935cacc7-8b0f-41b8-9cfa-3e3d7c3bc114 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:50.166307 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-935cacc7-8b0f-41b8-9cfa-3e3d7c3bc114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:50.180729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ef96e484334796840f8c8ff1aeb887'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:50.182456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ef96e484334796840f8c8ff1aeb887'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:50.214854 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ef96e484334796840f8c8ff1aeb887'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:50.216248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d9afdfa9-d7b6-4dad-b55b-7751f5a57644 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ef96e484334796840f8c8ff1aeb887'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:50.238755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1323/2643] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:49 2022] DELETE /identity/v3/users/b2ef96e484334796840f8c8ff1aeb887 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:50.238755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ea525-fe95-463f-a959-52073f876e62 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:50.238755 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ea525-fe95-463f-a959-52073f876e62 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:50.241434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ea525-fe95-463f-a959-52073f876e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:50.265336 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-c31ea525-fe95-463f-a959-52073f876e62 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:50.267078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c31ea525-fe95-463f-a959-52073f876e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:50.278678 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1321/2644] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:50.294052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c8043-9416-45c6-807e-16652b1b4586 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:50.294499 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c8043-9416-45c6-807e-16652b1b4586 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:50.294797 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c8043-9416-45c6-807e-16652b1b4586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:50.327084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d56c8043-9416-45c6-807e-16652b1b4586 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:50.327894 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d56c8043-9416-45c6-807e-16652b1b4586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:50.380210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1324/2645] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:50.392442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ea525-fe95-463f-a959-52073f876e62 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:50.394932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c31ea525-fe95-463f-a959-52073f876e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:50.416666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:50.439677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1322/2646] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:50.447250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c8043-9416-45c6-807e-16652b1b4586 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:50.458037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c8043-9416-45c6-807e-16652b1b4586 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:50.465192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3aa9b-74da-4a71-b1bd-2f70f3ae5ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:50.466130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3aa9b-74da-4a71-b1bd-2f70f3ae5ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:50.466548 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3aa9b-74da-4a71-b1bd-2f70f3ae5ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:50.466956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3aa9b-74da-4a71-b1bd-2f70f3ae5ac9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:50.469329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3aa9b-74da-4a71-b1bd-2f70f3ae5ac9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:37:50.469329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3aa9b-74da-4a71-b1bd-2f70f3ae5ac9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:50.471476 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3aa9b-74da-4a71-b1bd-2f70f3ae5ac9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:50.489186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1323/2647] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:37:50.494316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ea525-fe95-463f-a959-52073f876e62 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:50.496464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3aa9b-74da-4a71-b1bd-2f70f3ae5ac9 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:50.501462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffce2e-6363-4cb1-a26b-fd1269218758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:50.502102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffce2e-6363-4cb1-a26b-fd1269218758 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:50.502396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffce2e-6363-4cb1-a26b-fd1269218758 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:50.502663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffce2e-6363-4cb1-a26b-fd1269218758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:50.517354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffce2e-6363-4cb1-a26b-fd1269218758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:50.518481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3aa9b-74da-4a71-b1bd-2f70f3ae5ac9 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:50.519087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ffce2e-6363-4cb1-a26b-fd1269218758 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:50.531990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fa789-1b5e-4fe6-b73e-adb41279d0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:50.532919 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fa789-1b5e-4fe6-b73e-adb41279d0b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:50.533372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fa789-1b5e-4fe6-b73e-adb41279d0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:50.533835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782fa789-1b5e-4fe6-b73e-adb41279d0b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:50.534825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1325/2648] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:37:50.535085 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fa789-1b5e-4fe6-b73e-adb41279d0b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:50.538643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782fa789-1b5e-4fe6-b73e-adb41279d0b6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:50.735841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1324/2649] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:50 2022] POST /identity/v3/projects => generated 381 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:37:50.741795 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffce2e-6363-4cb1-a26b-fd1269218758 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:50.748276 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:50.756159 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffce2e-6363-4cb1-a26b-fd1269218758 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:50.844527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f0e23-0b08-43dc-84c5-3cf8d791cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:50.847032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f0e23-0b08-43dc-84c5-3cf8d791cb71 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:50.847689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f0e23-0b08-43dc-84c5-3cf8d791cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:50.848069 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f0e23-0b08-43dc-84c5-3cf8d791cb71 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:50.849314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f0e23-0b08-43dc-84c5-3cf8d791cb71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:50.854166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f0e23-0b08-43dc-84c5-3cf8d791cb71 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:50.989778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1326/2650] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:50 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:50.990967 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fa789-1b5e-4fe6-b73e-adb41279d0b6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:51.010025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782fa789-1b5e-4fe6-b73e-adb41279d0b6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:51.016435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a6afc-db0f-466c-943a-b94cda85e26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:51.017223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a6afc-db0f-466c-943a-b94cda85e26e admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:51.017585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a6afc-db0f-466c-943a-b94cda85e26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:51.017858 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a6afc-db0f-466c-943a-b94cda85e26e admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:51.018613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a6afc-db0f-466c-943a-b94cda85e26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:51.020471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a6afc-db0f-466c-943a-b94cda85e26e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:51.034140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1325/2651] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:51.039164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f0e23-0b08-43dc-84c5-3cf8d791cb71 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:51.054946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f0e23-0b08-43dc-84c5-3cf8d791cb71 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:51.063255 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128071a-9dc8-47c9-a639-791e564eb48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:51.064270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128071a-9dc8-47c9-a639-791e564eb48e admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:51.064558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128071a-9dc8-47c9-a639-791e564eb48e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:51.064834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128071a-9dc8-47c9-a639-791e564eb48e admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:51.065558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128071a-9dc8-47c9-a639-791e564eb48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:51.067291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128071a-9dc8-47c9-a639-791e564eb48e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:51.080186 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1327/2652] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:51.084631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a6afc-db0f-466c-943a-b94cda85e26e admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:51.095954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a6afc-db0f-466c-943a-b94cda85e26e admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:51.102319 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc4870-57ce-42d0-a825-e321d9c03e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:51.103683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc4870-57ce-42d0-a825-e321d9c03e52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:51.104022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc4870-57ce-42d0-a825-e321d9c03e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:51.108994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc4870-57ce-42d0-a825-e321d9c03e52 admin admin] PATH_INFO: `/v3/projects/8be1fb8ffa4445fe872cf915ab18335a/users/a0f3e4e42c064304b20a2582087403a0/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:51.123643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc4870-57ce-42d0-a825-e321d9c03e52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8be1fb8ffa4445fe872cf915ab18335a, user_id=a0f3e4e42c064304b20a2582087403a0, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:51.126616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc4870-57ce-42d0-a825-e321d9c03e52 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:51.137156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128071a-9dc8-47c9-a639-791e564eb48e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:51.151082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-5128071a-9dc8-47c9-a639-791e564eb48e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:51.159059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128071a-9dc8-47c9-a639-791e564eb48e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:51.171266 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:51.172489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:51.173014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:51.173516 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] PATH_INFO: `/v3/projects/5b20c1a0b392452b8ca1a6813ea18cbb` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:51.175909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b20c1a0b392452b8ca1a6813ea18cbb)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:51.180544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:51.226755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1326/2653] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:51 2022] PUT /identity/v3/projects/8be1fb8ffa4445fe872cf915ab18335a/users/a0f3e4e42c064304b20a2582087403a0/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:51.242596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f75d0-c2ae-463e-8c4a-deb9ed5f3ab4 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:51.243671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f75d0-c2ae-463e-8c4a-deb9ed5f3ab4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:51.243956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f75d0-c2ae-463e-8c4a-deb9ed5f3ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:51.291674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b86f75d0-c2ae-463e-8c4a-deb9ed5f3ab4 None None] MFA Rules not processed for user `a0f3e4e42c064304b20a2582087403a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:51.293653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b86f75d0-c2ae-463e-8c4a-deb9ed5f3ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:51.361345 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b20c1a0b392452b8ca1a6813ea18cbb', 'request_id': 'req-51d129df-d0e1-497d-955c-a5fc09373f69'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:51.375987 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b20c1a0b392452b8ca1a6813ea18cbb', 'request_id': 'req-51d129df-d0e1-497d-955c-a5fc09373f69'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:51.425805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1327/2654] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:51.432965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f75d0-c2ae-463e-8c4a-deb9ed5f3ab4 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:51.486659 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b20c1a0b392452b8ca1a6813ea18cbb', 'request_id': 'req-51d129df-d0e1-497d-955c-a5fc09373f69'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:51.503284 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] The token cache is being invalidate because project 5b20c1a0b392452b8ca1a6813ea18cbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:51.503625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:51.504374 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:51.506821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1328/2655] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:51 2022] DELETE /identity/v3/projects/5b20c1a0b392452b8ca1a6813ea18cbb => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:51.512926 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:51.515409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:51.545847 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f75d0-c2ae-463e-8c4a-deb9ed5f3ab4 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:51.564181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:51.615706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d129df-d0e1-497d-955c-a5fc09373f69 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:51.620390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9cbc6-2345-4337-8b51-4ef8c33f519f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:51.621396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cbc6-2345-4337-8b51-4ef8c33f519f admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:51.621615 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:51.621869 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cbc6-2345-4337-8b51-4ef8c33f519f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:51.622249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cbc6-2345-4337-8b51-4ef8c33f519f admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:51.622425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:51.622760 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:51.623084 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] PATH_INFO: `/v3/users/efcb8153b8a2426a9acbae87fe774e78` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:51.623379 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9cbc6-2345-4337-8b51-4ef8c33f519f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:51.625264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efcb8153b8a2426a9acbae87fe774e78)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:51.627373 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:51.627879 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9cbc6-2345-4337-8b51-4ef8c33f519f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:51.638375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:51.760901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1328/2656] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:51 2022] POST /identity/v3/projects => generated 381 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:51.761550 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9cbc6-2345-4337-8b51-4ef8c33f519f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:51.763702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-aba9cbc6-2345-4337-8b51-4ef8c33f519f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:51.849443 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9cbc6-2345-4337-8b51-4ef8c33f519f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:51.855289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8539ca8-77b2-42ab-a72e-8b7890cb6088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:51.856246 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8539ca8-77b2-42ab-a72e-8b7890cb6088 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:51.856654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8539ca8-77b2-42ab-a72e-8b7890cb6088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:51.857073 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8539ca8-77b2-42ab-a72e-8b7890cb6088 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:51.857897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8539ca8-77b2-42ab-a72e-8b7890cb6088 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:51.860407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8539ca8-77b2-42ab-a72e-8b7890cb6088 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:51.979260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efcb8153b8a2426a9acbae87fe774e78'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:52.003417 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1329/2657] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:51 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:37:52.010238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8539ca8-77b2-42ab-a72e-8b7890cb6088 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:52.023533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8539ca8-77b2-42ab-a72e-8b7890cb6088 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:52.031296 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808957c8-c23f-4ff4-8e57-29a56f8d71e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:52.036709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808957c8-c23f-4ff4-8e57-29a56f8d71e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:52.036709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808957c8-c23f-4ff4-8e57-29a56f8d71e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:52.036709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808957c8-c23f-4ff4-8e57-29a56f8d71e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:52.036709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808957c8-c23f-4ff4-8e57-29a56f8d71e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:52.036709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808957c8-c23f-4ff4-8e57-29a56f8d71e2 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:52.050201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1330/2658] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:37:52.058723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808957c8-c23f-4ff4-8e57-29a56f8d71e2 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:52.064272 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efcb8153b8a2426a9acbae87fe774e78'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:52.065403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efcb8153b8a2426a9acbae87fe774e78'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:52.075079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808957c8-c23f-4ff4-8e57-29a56f8d71e2 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:52.080855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5719-2859-4349-a907-67a78789f387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:52.081547 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5719-2859-4349-a907-67a78789f387 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:52.081820 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5719-2859-4349-a907-67a78789f387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:52.082104 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f5719-2859-4349-a907-67a78789f387 admin admin] PATH_INFO: `/v3/projects/db71b26063a444f395312231b3d84a84/users/7cda973fc1e8447b9c9b1d4b79bc2154/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:52.088585 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efcb8153b8a2426a9acbae87fe774e78'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:52.089689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efcb8153b8a2426a9acbae87fe774e78'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:52.101215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f5719-2859-4349-a907-67a78789f387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db71b26063a444f395312231b3d84a84, user_id=7cda973fc1e8447b9c9b1d4b79bc2154, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:52.104734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f5719-2859-4349-a907-67a78789f387 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:52.112918 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1329/2659] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:51 2022] DELETE /identity/v3/users/efcb8153b8a2426a9acbae87fe774e78 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:52.229564 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1331/2660] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:52 2022] PUT /identity/v3/projects/db71b26063a444f395312231b3d84a84/users/7cda973fc1e8447b9c9b1d4b79bc2154/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:52.235924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:52.238352 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:52.346118 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983c85d-964c-40dd-aedb-1501369f34e9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:52.423395 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766214b-5acc-41cb-a5fb-29cebd9b7c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:52.424510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766214b-5acc-41cb-a5fb-29cebd9b7c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:52.425005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766214b-5acc-41cb-a5fb-29cebd9b7c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:52.425497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766214b-5acc-41cb-a5fb-29cebd9b7c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:52.426741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766214b-5acc-41cb-a5fb-29cebd9b7c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:52.429886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766214b-5acc-41cb-a5fb-29cebd9b7c32 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:52.447728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1330/2661] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:52 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:52.453654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5719-2859-4349-a907-67a78789f387 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:52.469680 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f5719-2859-4349-a907-67a78789f387 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:52.475095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742c4a9-eb01-41ae-aae6-7ce83258c328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:52.476021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742c4a9-eb01-41ae-aae6-7ce83258c328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:52.476369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742c4a9-eb01-41ae-aae6-7ce83258c328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:52.476747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742c4a9-eb01-41ae-aae6-7ce83258c328 admin admin] PATH_INFO: `/v3/projects/db71b26063a444f395312231b3d84a84/users/7cda973fc1e8447b9c9b1d4b79bc2154/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:52.478931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742c4a9-eb01-41ae-aae6-7ce83258c328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db71b26063a444f395312231b3d84a84, user_id=7cda973fc1e8447b9c9b1d4b79bc2154, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:52.482287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742c4a9-eb01-41ae-aae6-7ce83258c328 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:52.505865 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1332/2662] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:52 2022] PUT /identity/v3/projects/db71b26063a444f395312231b3d84a84/users/7cda973fc1e8447b9c9b1d4b79bc2154/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:52.508491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b3cb-2d8b-4813-9d8b-c6b5b9d526d5 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:52.509532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b3cb-2d8b-4813-9d8b-c6b5b9d526d5 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:52.510014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867b3cb-2d8b-4813-9d8b-c6b5b9d526d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:52.548791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-2867b3cb-2d8b-4813-9d8b-c6b5b9d526d5 None None] MFA Rules not processed for user `7cda973fc1e8447b9c9b1d4b79bc2154`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:52.550219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2867b3cb-2d8b-4813-9d8b-c6b5b9d526d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:52.688734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1331/2663] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:52.701599 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742c4a9-eb01-41ae-aae6-7ce83258c328 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:52.703614 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1742c4a9-eb01-41ae-aae6-7ce83258c328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:52.793923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867b3cb-2d8b-4813-9d8b-c6b5b9d526d5 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:52.795829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2867b3cb-2d8b-4813-9d8b-c6b5b9d526d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:52.801816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:52.820948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742c4a9-eb01-41ae-aae6-7ce83258c328 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:52.832915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecbd9f-4d13-4e40-9251-16ba84285896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:52.836558 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecbd9f-4d13-4e40-9251-16ba84285896 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:52.837123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecbd9f-4d13-4e40-9251-16ba84285896 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:52.837411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecbd9f-4d13-4e40-9251-16ba84285896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:52.860001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecbd9f-4d13-4e40-9251-16ba84285896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:52.861641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecbd9f-4d13-4e40-9251-16ba84285896 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:52.874661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1333/2664] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:37:52.919071 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867b3cb-2d8b-4813-9d8b-c6b5b9d526d5 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:52.926017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:52.927103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:52.927396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:52.927724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] PATH_INFO: `/v3/projects/28bde6da6d634c37925bd7cddd0c2720` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:52.928959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28bde6da6d634c37925bd7cddd0c2720)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:52.931503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:52.945716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:53.122586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28bde6da6d634c37925bd7cddd0c2720', 'request_id': 'req-8d16db08-71af-4ea1-b1bf-046f4c0fef96'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:53.137433 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28bde6da6d634c37925bd7cddd0c2720', 'request_id': 'req-8d16db08-71af-4ea1-b1bf-046f4c0fef96'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:53.273649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28bde6da6d634c37925bd7cddd0c2720', 'request_id': 'req-8d16db08-71af-4ea1-b1bf-046f4c0fef96'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:53.298848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] The token cache is being invalidate because project 28bde6da6d634c37925bd7cddd0c2720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:53.299309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:53.300663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d16db08-71af-4ea1-b1bf-046f4c0fef96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:53.303632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1332/2665] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:52 2022] DELETE /identity/v3/projects/28bde6da6d634c37925bd7cddd0c2720 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:53.312108 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f7ec5-7f7b-4314-b04e-f1c7d14d7de7 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:53.312613 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f7ec5-7f7b-4314-b04e-f1c7d14d7de7 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:53.313134 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f7ec5-7f7b-4314-b04e-f1c7d14d7de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:53.350909 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-983f7ec5-7f7b-4314-b04e-f1c7d14d7de7 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:53.352201 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-983f7ec5-7f7b-4314-b04e-f1c7d14d7de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:53.437666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1334/2666] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:53.453139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640ae5d-752f-4ec1-a4a7-e86364915a33 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:53.453990 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640ae5d-752f-4ec1-a4a7-e86364915a33 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:53.455422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640ae5d-752f-4ec1-a4a7-e86364915a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:53.494135 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a640ae5d-752f-4ec1-a4a7-e86364915a33 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:53.496292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a640ae5d-752f-4ec1-a4a7-e86364915a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:53.510055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1333/2667] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:53.520084 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f7ec5-7f7b-4314-b04e-f1c7d14d7de7 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:53.537898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f7ec5-7f7b-4314-b04e-f1c7d14d7de7 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:53.546313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05af02-6cae-4d3d-bd2a-3528756ead73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:53.547287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05af02-6cae-4d3d-bd2a-3528756ead73 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:53.547844 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05af02-6cae-4d3d-bd2a-3528756ead73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:53.548364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05af02-6cae-4d3d-bd2a-3528756ead73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:53.549582 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05af02-6cae-4d3d-bd2a-3528756ead73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:37:53.550161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05af02-6cae-4d3d-bd2a-3528756ead73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:53.554411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05af02-6cae-4d3d-bd2a-3528756ead73 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:53.576497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1335/2668] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:37:53.582212 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640ae5d-752f-4ec1-a4a7-e86364915a33 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:53.595414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640ae5d-752f-4ec1-a4a7-e86364915a33 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:53.602220 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8700a-f466-4116-b162-ab0c73564edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:53.609746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8700a-f466-4116-b162-ab0c73564edf admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:53.609746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8700a-f466-4116-b162-ab0c73564edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:53.609746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b8700a-f466-4116-b162-ab0c73564edf admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:53.609746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8700a-f466-4116-b162-ab0c73564edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:53.609746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b8700a-f466-4116-b162-ab0c73564edf admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:53.623181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05af02-6cae-4d3d-bd2a-3528756ead73 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:53.624994 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6c05af02-6cae-4d3d-bd2a-3528756ead73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:53.638729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05af02-6cae-4d3d-bd2a-3528756ead73 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:53.643734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:53.644400 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:53.644683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:53.644965 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] PATH_INFO: `/v3/users/a0f3e4e42c064304b20a2582087403a0` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:53.646204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f3e4e42c064304b20a2582087403a0)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:53.648236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:53.760763 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1334/2669] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:53 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:37:53.765735 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8700a-f466-4116-b162-ab0c73564edf admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:53.866091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b8700a-f466-4116-b162-ab0c73564edf admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:53.869236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f3e4e42c064304b20a2582087403a0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:53.870636 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f3e4e42c064304b20a2582087403a0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:53.871578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f3e4e42c064304b20a2582087403a0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:53.875293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:53.879980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af2e3e-8dc9-4679-850b-b6335809f035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:53.881061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af2e3e-8dc9-4679-850b-b6335809f035 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:53.881665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af2e3e-8dc9-4679-850b-b6335809f035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:53.882190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af2e3e-8dc9-4679-850b-b6335809f035 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:53.883503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af2e3e-8dc9-4679-850b-b6335809f035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:53.883774 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f3e4e42c064304b20a2582087403a0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:53.887372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af2e3e-8dc9-4679-850b-b6335809f035 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:53.903623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-0dc67445-09d7-4b58-9979-9d7ba86e605e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f3e4e42c064304b20a2582087403a0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:54.038994 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1335/2670] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:53 2022] POST /identity/v3/users => generated 284 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:37:54.044191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af2e3e-8dc9-4679-850b-b6335809f035 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:54.046413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e4af2e3e-8dc9-4679-850b-b6335809f035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:54.062900 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af2e3e-8dc9-4679-850b-b6335809f035 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:54.068444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3db80b-f607-4fc7-a06a-f3c4a3b3c914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:54.069093 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3db80b-f607-4fc7-a06a-f3c4a3b3c914 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:54.069886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3db80b-f607-4fc7-a06a-f3c4a3b3c914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:54.070178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3db80b-f607-4fc7-a06a-f3c4a3b3c914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:54.070609 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1336/2671] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:53 2022] DELETE /identity/v3/users/a0f3e4e42c064304b20a2582087403a0 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:54.071039 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3db80b-f607-4fc7-a06a-f3c4a3b3c914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:54.073527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3db80b-f607-4fc7-a06a-f3c4a3b3c914 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:54.076273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1e114-175c-4ec6-a2aa-6e36729fb645 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:54.076552 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1e114-175c-4ec6-a2aa-6e36729fb645 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:54.076837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1e114-175c-4ec6-a2aa-6e36729fb645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:54.088957 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1336/2672] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:37:54.095399 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3db80b-f607-4fc7-a06a-f3c4a3b3c914 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:54.102391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-61a1e114-175c-4ec6-a2aa-6e36729fb645 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:54.103951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-61a1e114-175c-4ec6-a2aa-6e36729fb645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:54.109770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3db80b-f607-4fc7-a06a-f3c4a3b3c914 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:54.164019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:54.190571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1337/2673] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:54.199400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bce6ec-40e5-4064-b402-781153ac2bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:54.200356 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bce6ec-40e5-4064-b402-781153ac2bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:54.200695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bce6ec-40e5-4064-b402-781153ac2bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:54.201129 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bce6ec-40e5-4064-b402-781153ac2bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:54.202081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bce6ec-40e5-4064-b402-781153ac2bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:54.204213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bce6ec-40e5-4064-b402-781153ac2bc6 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:54.206490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1e114-175c-4ec6-a2aa-6e36729fb645 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:54.208978 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-61a1e114-175c-4ec6-a2aa-6e36729fb645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:54.223364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1337/2674] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:37:54.230596 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bce6ec-40e5-4064-b402-781153ac2bc6 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:54.246598 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bce6ec-40e5-4064-b402-781153ac2bc6 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:54.254470 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce1473-5267-4dc1-9fdc-3dc331a0f135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:54.255209 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce1473-5267-4dc1-9fdc-3dc331a0f135 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:54.255509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce1473-5267-4dc1-9fdc-3dc331a0f135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:54.255867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce1473-5267-4dc1-9fdc-3dc331a0f135 admin admin] PATH_INFO: `/v3/projects/67af3d29165e46288763e578d87ffbb2/users/8e7b9d2e98b44681a091c37e888fa867/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:54.278141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce1473-5267-4dc1-9fdc-3dc331a0f135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67af3d29165e46288763e578d87ffbb2, user_id=8e7b9d2e98b44681a091c37e888fa867, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:54.281213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce1473-5267-4dc1-9fdc-3dc331a0f135 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:54.296106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1e114-175c-4ec6-a2aa-6e36729fb645 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:54.300284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d4b32-5145-4d1c-81dc-4e12ce1ee00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:54.300906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d4b32-5145-4d1c-81dc-4e12ce1ee00e service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:54.301209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d4b32-5145-4d1c-81dc-4e12ce1ee00e service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:54.301488 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d4b32-5145-4d1c-81dc-4e12ce1ee00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:54.312622 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d4b32-5145-4d1c-81dc-4e12ce1ee00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:54.314294 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d4b32-5145-4d1c-81dc-4e12ce1ee00e service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:54.325676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1338/2675] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:54.365007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1338/2676] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:54 2022] PUT /identity/v3/projects/67af3d29165e46288763e578d87ffbb2/users/8e7b9d2e98b44681a091c37e888fa867/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:54.471088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fbe60-0203-4ea6-b239-8b60bcddbc15 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:54.471747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fbe60-0203-4ea6-b239-8b60bcddbc15 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:54.472299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fbe60-0203-4ea6-b239-8b60bcddbc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:54.503904 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-cb0fbe60-0203-4ea6-b239-8b60bcddbc15 None None] MFA Rules not processed for user `8e7b9d2e98b44681a091c37e888fa867`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:54.505131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-cb0fbe60-0203-4ea6-b239-8b60bcddbc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:54.589982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1339/2677] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:54.599222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce1473-5267-4dc1-9fdc-3dc331a0f135 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:54.601381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-56ce1473-5267-4dc1-9fdc-3dc331a0f135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:54.680653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce1473-5267-4dc1-9fdc-3dc331a0f135 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:54.685543 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5071b-9e86-4d88-a6f3-443cce18e9c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f10c258fe5f45ccab1669fa32ed6029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:54.686293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5071b-9e86-4d88-a6f3-443cce18e9c1 service glance] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:54.686785 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5071b-9e86-4d88-a6f3-443cce18e9c1 service glance] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:54.687529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5071b-9e86-4d88-a6f3-443cce18e9c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:54.698304 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5071b-9e86-4d88-a6f3-443cce18e9c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:54.699988 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5071b-9e86-4d88-a6f3-443cce18e9c1 service glance] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:54.711789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1339/2678] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:37:54.873369 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fbe60-0203-4ea6-b239-8b60bcddbc15 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:54.875089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-cb0fbe60-0203-4ea6-b239-8b60bcddbc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:54.978743 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fbe60-0203-4ea6-b239-8b60bcddbc15 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:54.983607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:54.984431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:54.984815 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:54.985164 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] PATH_INFO: `/v3/projects/8be1fb8ffa4445fe872cf915ab18335a` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:54.986407 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8be1fb8ffa4445fe872cf915ab18335a)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:54.988562 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:55.017719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=100 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:55.196831 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8be1fb8ffa4445fe872cf915ab18335a', 'request_id': 'req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:55.198462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8be1fb8ffa4445fe872cf915ab18335a', 'request_id': 'req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:55.373090 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8be1fb8ffa4445fe872cf915ab18335a', 'request_id': 'req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:55.387968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] The token cache is being invalidate because project 8be1fb8ffa4445fe872cf915ab18335a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:55.388242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:55.388920 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:55.390668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1340/2679] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:54 2022] DELETE /identity/v3/projects/8be1fb8ffa4445fe872cf915ab18335a => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:55.395282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5071b-9e86-4d88-a6f3-443cce18e9c1 service glance] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:55.408252 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5071b-9e86-4d88-a6f3-443cce18e9c1 service glance] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:55.413217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:55.499484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:55.500664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:55.500664 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:55.500932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] PATH_INFO: `/v3/users/7cda973fc1e8447b9c9b1d4b79bc2154` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:55.502205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cda973fc1e8447b9c9b1d4b79bc2154)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:55.504416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:55.750573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cda973fc1e8447b9c9b1d4b79bc2154'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:55.841194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cda973fc1e8447b9c9b1d4b79bc2154'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:55.844178 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cda973fc1e8447b9c9b1d4b79bc2154'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:55.876024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cda973fc1e8447b9c9b1d4b79bc2154'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:55.876024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cda973fc1e8447b9c9b1d4b79bc2154'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:55.892640 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:55.895015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:55.897076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1340/2680] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:37:55 2022] DELETE /identity/v3/users/7cda973fc1e8447b9c9b1d4b79bc2154 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:37:55.987727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:55.989833 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:56.055055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbdda3-08fd-4a4a-a882-2a4a9a1ac8c6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:56.102857 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa7a651-8ead-4617-9cc5-4c7488c89f65 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:56.108817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c90206-56d1-4066-9279-fe0090305de3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:56.109927 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c90206-56d1-4066-9279-fe0090305de3 service placement] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:56.110131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c90206-56d1-4066-9279-fe0090305de3 service placement] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:56.110602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c90206-56d1-4066-9279-fe0090305de3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:56.114017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b5c90206-56d1-4066-9279-fe0090305de3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:56.130199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c90206-56d1-4066-9279-fe0090305de3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:56.132507 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c90206-56d1-4066-9279-fe0090305de3 service placement] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:56.147500 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1341/2681] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:56.157169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be474b3-0286-419e-9ced-62b04d32d981 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:56.157923 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be474b3-0286-419e-9ced-62b04d32d981 service placement] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:56.158302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be474b3-0286-419e-9ced-62b04d32d981 service placement] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:56.158625 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be474b3-0286-419e-9ced-62b04d32d981 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:56.172643 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be474b3-0286-419e-9ced-62b04d32d981 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:56.174074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be474b3-0286-419e-9ced-62b04d32d981 service placement] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:56.190095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1341/2682] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:56.669106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c90206-56d1-4066-9279-fe0090305de3 service placement] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:56.672499 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b5c90206-56d1-4066-9279-fe0090305de3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:56.715931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:56.771773 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c90206-56d1-4066-9279-fe0090305de3 service placement] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:56.777557 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:56.778223 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:56.778510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:56.778784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] PATH_INFO: `/v3/projects/db71b26063a444f395312231b3d84a84` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:56.780027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db71b26063a444f395312231b3d84a84)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:56.782085 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:56.930408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db71b26063a444f395312231b3d84a84', 'request_id': 'req-d3fc81a0-3c00-424e-988b-dd7875348333'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:56.937960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db71b26063a444f395312231b3d84a84', 'request_id': 'req-d3fc81a0-3c00-424e-988b-dd7875348333'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:57.071196 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db71b26063a444f395312231b3d84a84', 'request_id': 'req-d3fc81a0-3c00-424e-988b-dd7875348333'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:57.088743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] The token cache is being invalidate because project db71b26063a444f395312231b3d84a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:37:57.089152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:57.089886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-d3fc81a0-3c00-424e-988b-dd7875348333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:37:57.091630 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1342/2683] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:37:56 2022] DELETE /identity/v3/projects/db71b26063a444f395312231b3d84a84 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:57.100126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db202e-0030-458c-82cd-fe00bab4e08b None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:57.101380 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db202e-0030-458c-82cd-fe00bab4e08b None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:57.101991 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db202e-0030-458c-82cd-fe00bab4e08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:57.130035 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-d0db202e-0030-458c-82cd-fe00bab4e08b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:57.131287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-d0db202e-0030-458c-82cd-fe00bab4e08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:57.154189 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:57.210078 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1342/2684] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:57.220820 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf4d84-8e8b-4072-985f-c176a0cbaa24 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:57.221103 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf4d84-8e8b-4072-985f-c176a0cbaa24 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:57.221377 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cf4d84-8e8b-4072-985f-c176a0cbaa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:57.374817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-53cf4d84-8e8b-4072-985f-c176a0cbaa24 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:57.376351 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-53cf4d84-8e8b-4072-985f-c176a0cbaa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:57.384935 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1343/2685] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:37:57.389979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db202e-0030-458c-82cd-fe00bab4e08b None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:57.401188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db202e-0030-458c-82cd-fe00bab4e08b None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:57.406110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24509439-68cc-4c3e-9e9e-bf41592f66c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:57.406741 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24509439-68cc-4c3e-9e9e-bf41592f66c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:57.407042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24509439-68cc-4c3e-9e9e-bf41592f66c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:57.407337 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24509439-68cc-4c3e-9e9e-bf41592f66c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:57.407993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24509439-68cc-4c3e-9e9e-bf41592f66c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:37:57.408411 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24509439-68cc-4c3e-9e9e-bf41592f66c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:57.410224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24509439-68cc-4c3e-9e9e-bf41592f66c6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:57.430966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1343/2686] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:37:57.435762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf4d84-8e8b-4072-985f-c176a0cbaa24 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:57.447097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cf4d84-8e8b-4072-985f-c176a0cbaa24 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:57.452980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6eee5-2d7e-47fc-96d3-cf67937f1431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:57.453770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6eee5-2d7e-47fc-96d3-cf67937f1431 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:57.454049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6eee5-2d7e-47fc-96d3-cf67937f1431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:57.454375 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6eee5-2d7e-47fc-96d3-cf67937f1431 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:57.455060 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6eee5-2d7e-47fc-96d3-cf67937f1431 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:57.457059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6eee5-2d7e-47fc-96d3-cf67937f1431 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:57.637182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1344/2687] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:37:57 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:37:57.642441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24509439-68cc-4c3e-9e9e-bf41592f66c6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:57.660332 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24509439-68cc-4c3e-9e9e-bf41592f66c6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:57.787176 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6f351-c6c4-4c8b-98be-4a37f3e1ee94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:57.788458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6f351-c6c4-4c8b-98be-4a37f3e1ee94 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:57.789064 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6f351-c6c4-4c8b-98be-4a37f3e1ee94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:57.789519 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6f351-c6c4-4c8b-98be-4a37f3e1ee94 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:57.790708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6f351-c6c4-4c8b-98be-4a37f3e1ee94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:57.792975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6f351-c6c4-4c8b-98be-4a37f3e1ee94 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:57.937169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1344/2688] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:37:57 2022] POST /identity/v3/users => generated 289 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:37:57.943766 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6eee5-2d7e-47fc-96d3-cf67937f1431 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:57.956195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6eee5-2d7e-47fc-96d3-cf67937f1431 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:57.962316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf0a48-ce59-4aaf-9819-cf405e77d975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:57.964095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf0a48-ce59-4aaf-9819-cf405e77d975 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:57.964095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf0a48-ce59-4aaf-9819-cf405e77d975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:57.964095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf0a48-ce59-4aaf-9819-cf405e77d975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:57.964652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf0a48-ce59-4aaf-9819-cf405e77d975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:57.968627 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf0a48-ce59-4aaf-9819-cf405e77d975 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:57.989270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1345/2689] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:57.990408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6f351-c6c4-4c8b-98be-4a37f3e1ee94 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:58.010473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6f351-c6c4-4c8b-98be-4a37f3e1ee94 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:58.016184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea8d16-8355-44d1-84b3-e25d1f44b78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:58.017028 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea8d16-8355-44d1-84b3-e25d1f44b78e admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:58.017328 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea8d16-8355-44d1-84b3-e25d1f44b78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:58.017813 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea8d16-8355-44d1-84b3-e25d1f44b78e admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:58.018666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea8d16-8355-44d1-84b3-e25d1f44b78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:58.020390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea8d16-8355-44d1-84b3-e25d1f44b78e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:58.033780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1345/2690] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:37:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:37:58.038331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf0a48-ce59-4aaf-9819-cf405e77d975 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:58.054049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf0a48-ce59-4aaf-9819-cf405e77d975 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:58.068843 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a945f7-dcef-41eb-90c2-825d20627116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:58.069531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a945f7-dcef-41eb-90c2-825d20627116 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:58.069815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a945f7-dcef-41eb-90c2-825d20627116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:58.070096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a945f7-dcef-41eb-90c2-825d20627116 admin admin] PATH_INFO: `/v3/projects/f0cf90d3ed5d447fb2f29e745d6435ac/users/a2d7269796cc45a58a01565289763e9c/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:58.100521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a945f7-dcef-41eb-90c2-825d20627116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0cf90d3ed5d447fb2f29e745d6435ac, user_id=a2d7269796cc45a58a01565289763e9c, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:58.105954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a945f7-dcef-41eb-90c2-825d20627116 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:58.227225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1346/2691] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:37:58 2022] PUT /identity/v3/projects/f0cf90d3ed5d447fb2f29e745d6435ac/users/a2d7269796cc45a58a01565289763e9c/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:37:58.241171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dbfaa-c473-44a1-a588-f835046029af None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:58.241953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dbfaa-c473-44a1-a588-f835046029af None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:58.241953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dbfaa-c473-44a1-a588-f835046029af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:58.310074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-084dbfaa-c473-44a1-a588-f835046029af None None] MFA Rules not processed for user `a2d7269796cc45a58a01565289763e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:37:58.312855 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-084dbfaa-c473-44a1-a588-f835046029af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:58.463728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1346/2692] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:37:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:37:58.465882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a945f7-dcef-41eb-90c2-825d20627116 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:58.469925 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-39a945f7-dcef-41eb-90c2-825d20627116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:58.479127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dbfaa-c473-44a1-a588-f835046029af None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:58.485997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-084dbfaa-c473-44a1-a588-f835046029af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:58.523698 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:58.559532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:37:58.586553 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a945f7-dcef-41eb-90c2-825d20627116 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:58.600767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b5ae8-6780-41fb-8d66-0bb5f55d6a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:58.602036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5ae8-6780-41fb-8d66-0bb5f55d6a3e service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:58.602225 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5ae8-6780-41fb-8d66-0bb5f55d6a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:58.602412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b5ae8-6780-41fb-8d66-0bb5f55d6a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:58.604691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-278b5ae8-6780-41fb-8d66-0bb5f55d6a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:58.621569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b5ae8-6780-41fb-8d66-0bb5f55d6a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:58.623179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b5ae8-6780-41fb-8d66-0bb5f55d6a3e service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:58.627578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dbfaa-c473-44a1-a588-f835046029af None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:58.634788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d572-e302-4fe2-bee2-b1101af6fa77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:58.635107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1347/2693] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:37:58.636787 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d572-e302-4fe2-bee2-b1101af6fa77 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:58.638162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d572-e302-4fe2-bee2-b1101af6fa77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:58.638162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224d572-e302-4fe2-bee2-b1101af6fa77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:58.656825 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224d572-e302-4fe2-bee2-b1101af6fa77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:58.659300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224d572-e302-4fe2-bee2-b1101af6fa77 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:37:58.679434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1347/2694] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:37:59.776358 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b5ae8-6780-41fb-8d66-0bb5f55d6a3e service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:37:59.781215 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-278b5ae8-6780-41fb-8d66-0bb5f55d6a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:59.802526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b5ae8-6780-41fb-8d66-0bb5f55d6a3e service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:37:59.809915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692ddef-3ded-41f8-a90c-e4e046d8a351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:37:59.812125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ddef-3ded-41f8-a90c-e4e046d8a351 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:37:59.812430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ddef-3ded-41f8-a90c-e4e046d8a351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:37:59.812917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ddef-3ded-41f8-a90c-e4e046d8a351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:37:59.818634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b692ddef-3ded-41f8-a90c-e4e046d8a351 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:37:59.993147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692ddef-3ded-41f8-a90c-e4e046d8a351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:37:59.994914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b692ddef-3ded-41f8-a90c-e4e046d8a351 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:00.018575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1348/2695] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:37:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:38:08.670250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=18 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:10.015142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=53 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:12.451323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d572-e302-4fe2-bee2-b1101af6fa77 service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:12.462269 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224d572-e302-4fe2-bee2-b1101af6fa77 service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:12.467727 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310cf52-afbe-4247-a267-f3e8ad7d810d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:12.468397 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310cf52-afbe-4247-a267-f3e8ad7d810d service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:12.468765 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310cf52-afbe-4247-a267-f3e8ad7d810d service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:12.468957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310cf52-afbe-4247-a267-f3e8ad7d810d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:12.471883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-a310cf52-afbe-4247-a267-f3e8ad7d810d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:12.604285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310cf52-afbe-4247-a267-f3e8ad7d810d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:12.607142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310cf52-afbe-4247-a267-f3e8ad7d810d service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:12.625931 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1348/2696] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:38:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:38:17.394146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692ddef-3ded-41f8-a90c-e4e046d8a351 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:17.399419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b692ddef-3ded-41f8-a90c-e4e046d8a351 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:17.413283 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692ddef-3ded-41f8-a90c-e4e046d8a351 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:17.418876 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:17.419532 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:17.419841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:17.420114 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] PATH_INFO: `/v3/users/62f175cb5d1e4ac8a0ee95cf7718837b` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:17.421288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62f175cb5d1e4ac8a0ee95cf7718837b)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:17.423206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:17.652528 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62f175cb5d1e4ac8a0ee95cf7718837b'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:17.750227 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62f175cb5d1e4ac8a0ee95cf7718837b'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:17.750850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62f175cb5d1e4ac8a0ee95cf7718837b'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:17.767235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62f175cb5d1e4ac8a0ee95cf7718837b'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:17.768050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-75ee4ed3-5b5c-4d03-b0d4-d24c46532f29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62f175cb5d1e4ac8a0ee95cf7718837b'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:17.780353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1349/2697] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:38:17 2022] DELETE /identity/v3/users/62f175cb5d1e4ac8a0ee95cf7718837b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:18.061389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310cf52-afbe-4247-a267-f3e8ad7d810d service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:18.072276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310cf52-afbe-4247-a267-f3e8ad7d810d service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:18.141511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:18.142142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:18.142421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:18.142695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] PATH_INFO: `/v3/projects/cc926b4dede94c3b92441b0c610771f6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:18.144004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc926b4dede94c3b92441b0c610771f6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:18.145895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:18.289883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc926b4dede94c3b92441b0c610771f6', 'request_id': 'req-bd05762f-180a-4d59-8b9d-e9211d7830b8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:18.297037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc926b4dede94c3b92441b0c610771f6', 'request_id': 'req-bd05762f-180a-4d59-8b9d-e9211d7830b8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:18.388840 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc926b4dede94c3b92441b0c610771f6', 'request_id': 'req-bd05762f-180a-4d59-8b9d-e9211d7830b8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:18.402333 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] The token cache is being invalidate because project cc926b4dede94c3b92441b0c610771f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:38:18.402730 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:18.403473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-bd05762f-180a-4d59-8b9d-e9211d7830b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:18.405105 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1349/2698] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:38:18 2022] DELETE /identity/v3/projects/cc926b4dede94c3b92441b0c610771f6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:38:18.410393 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a0f90-b531-4b5f-ace8-b7f0dbf90721 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:18.411192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a0f90-b531-4b5f-ace8-b7f0dbf90721 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:18.411371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a0f90-b531-4b5f-ace8-b7f0dbf90721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:18.441513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-452a0f90-b531-4b5f-ace8-b7f0dbf90721 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:18.442767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-452a0f90-b531-4b5f-ace8-b7f0dbf90721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:18.507892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1350/2699] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:18.516304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a9a35-3d76-4534-8c3b-92f3930ba6ed None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:18.516732 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a9a35-3d76-4534-8c3b-92f3930ba6ed None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:18.517122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a9a35-3d76-4534-8c3b-92f3930ba6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:18.538160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-327a9a35-3d76-4534-8c3b-92f3930ba6ed None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:18.539395 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-327a9a35-3d76-4534-8c3b-92f3930ba6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:18.547569 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1350/2700] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:38:18.551964 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a0f90-b531-4b5f-ace8-b7f0dbf90721 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:18.561722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a0f90-b531-4b5f-ace8-b7f0dbf90721 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:18.566328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075d343-3703-4102-9014-d69f85f62f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:18.566932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075d343-3703-4102-9014-d69f85f62f4e admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:18.567216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075d343-3703-4102-9014-d69f85f62f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:18.567495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075d343-3703-4102-9014-d69f85f62f4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:18.568188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075d343-3703-4102-9014-d69f85f62f4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:38:18.568495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075d343-3703-4102-9014-d69f85f62f4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:18.570105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075d343-3703-4102-9014-d69f85f62f4e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:18.582868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1351/2701] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:38:18.588192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a9a35-3d76-4534-8c3b-92f3930ba6ed None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:18.598773 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a9a35-3d76-4534-8c3b-92f3930ba6ed None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:18.604886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792075b7-8a58-4f6f-b6a7-0a6ac38b4b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:18.605016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792075b7-8a58-4f6f-b6a7-0a6ac38b4b9e admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:18.605206 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792075b7-8a58-4f6f-b6a7-0a6ac38b4b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:18.605517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792075b7-8a58-4f6f-b6a7-0a6ac38b4b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:18.606117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792075b7-8a58-4f6f-b6a7-0a6ac38b4b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:18.608089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792075b7-8a58-4f6f-b6a7-0a6ac38b4b9e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:18.625780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:18.735023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1351/2702] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:38:18 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:38:18.739618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075d343-3703-4102-9014-d69f85f62f4e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:18.751166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075d343-3703-4102-9014-d69f85f62f4e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:18.816776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4c6fe-62d3-4126-bbac-2e0c9bed1b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:18.817454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4c6fe-62d3-4126-bbac-2e0c9bed1b3c admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:18.817734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4c6fe-62d3-4126-bbac-2e0c9bed1b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:18.818010 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4c6fe-62d3-4126-bbac-2e0c9bed1b3c admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:18.818661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4c6fe-62d3-4126-bbac-2e0c9bed1b3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:18.820640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4c6fe-62d3-4126-bbac-2e0c9bed1b3c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:18.829432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:18.956096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1352/2703] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:38:18 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:38:18.960481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792075b7-8a58-4f6f-b6a7-0a6ac38b4b9e admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:18.974074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792075b7-8a58-4f6f-b6a7-0a6ac38b4b9e admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:18.980211 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e2257-bc7f-48bf-b0a9-aa9525d8048f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:18.980842 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e2257-bc7f-48bf-b0a9-aa9525d8048f admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:18.981127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e2257-bc7f-48bf-b0a9-aa9525d8048f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:18.981404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e2257-bc7f-48bf-b0a9-aa9525d8048f admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:18.982266 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e2257-bc7f-48bf-b0a9-aa9525d8048f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:18.984050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e2257-bc7f-48bf-b0a9-aa9525d8048f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:18.995656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1352/2704] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:38:19.000161 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4c6fe-62d3-4126-bbac-2e0c9bed1b3c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:19.011302 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4c6fe-62d3-4126-bbac-2e0c9bed1b3c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:19.016260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f4edc-9bc9-4440-a4d1-531ce1e2a910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:19.016873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f4edc-9bc9-4440-a4d1-531ce1e2a910 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:19.017156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f4edc-9bc9-4440-a4d1-531ce1e2a910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:19.017436 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f4edc-9bc9-4440-a4d1-531ce1e2a910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:19.018461 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f4edc-9bc9-4440-a4d1-531ce1e2a910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:19.020563 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f4edc-9bc9-4440-a4d1-531ce1e2a910 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:19.032681 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1353/2705] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:38:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:38:19.038861 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e2257-bc7f-48bf-b0a9-aa9525d8048f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:19.050162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e2257-bc7f-48bf-b0a9-aa9525d8048f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:19.055068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143efe2d-f345-4724-858b-79bc99982f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:19.055779 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143efe2d-f345-4724-858b-79bc99982f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:19.056065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143efe2d-f345-4724-858b-79bc99982f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:19.056478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143efe2d-f345-4724-858b-79bc99982f82 admin admin] PATH_INFO: `/v3/projects/a15460898c2b4b15bfee3d39a01273b4/users/3ce3e8c1e4f84274a1f5edb940ea4454/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:19.072256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143efe2d-f345-4724-858b-79bc99982f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15460898c2b4b15bfee3d39a01273b4, user_id=3ce3e8c1e4f84274a1f5edb940ea4454, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:19.074978 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143efe2d-f345-4724-858b-79bc99982f82 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:19.186947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1353/2706] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:38:19 2022] PUT /identity/v3/projects/a15460898c2b4b15bfee3d39a01273b4/users/3ce3e8c1e4f84274a1f5edb940ea4454/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:38:19.194793 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03041b77-7d44-453d-8b68-ca1d59a2f39d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:19.195256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03041b77-7d44-453d-8b68-ca1d59a2f39d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:19.195366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03041b77-7d44-453d-8b68-ca1d59a2f39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:19.215827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-03041b77-7d44-453d-8b68-ca1d59a2f39d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:19.217059 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-03041b77-7d44-453d-8b68-ca1d59a2f39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:19.285804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1354/2707] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:19.296656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143efe2d-f345-4724-858b-79bc99982f82 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:19.298568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-143efe2d-f345-4724-858b-79bc99982f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:19.362228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143efe2d-f345-4724-858b-79bc99982f82 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:19.366431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b304ab4-c98d-4804-b5a3-2b92064f1e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:19.370219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b304ab4-c98d-4804-b5a3-2b92064f1e1f service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:19.370219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b304ab4-c98d-4804-b5a3-2b92064f1e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:19.370219 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b304ab4-c98d-4804-b5a3-2b92064f1e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:19.378245 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b304ab4-c98d-4804-b5a3-2b92064f1e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:19.379728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b304ab4-c98d-4804-b5a3-2b92064f1e1f service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:19.389762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1354/2708] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:38:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:38:26.886609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6b72-1710-48bd-bbbc-f8ca00d24ba1 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:26.887351 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6b72-1710-48bd-bbbc-f8ca00d24ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:26.887591 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d6b72-1710-48bd-bbbc-f8ca00d24ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:26.921498 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f74d6b72-1710-48bd-bbbc-f8ca00d24ba1 None None] MFA Rules not processed for user `3ce3e8c1e4f84274a1f5edb940ea4454`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:26.923033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f74d6b72-1710-48bd-bbbc-f8ca00d24ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:27.000229 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1355/2709] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:38:27.018050 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b304ab4-c98d-4804-b5a3-2b92064f1e1f service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:27.019430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6b304ab4-c98d-4804-b5a3-2b92064f1e1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:27.123523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b304ab4-c98d-4804-b5a3-2b92064f1e1f service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:27.130655 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee72009-c23a-42be-a938-0ee6c6349be4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:27.131365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee72009-c23a-42be-a938-0ee6c6349be4 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:27.131723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee72009-c23a-42be-a938-0ee6c6349be4 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:27.138142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee72009-c23a-42be-a938-0ee6c6349be4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:27.141823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee72009-c23a-42be-a938-0ee6c6349be4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:27.143454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee72009-c23a-42be-a938-0ee6c6349be4 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:27.154274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1355/2710] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:38:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:38:30.564529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387f2ca-7172-428b-a5d8-634660d84350 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:30.565074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387f2ca-7172-428b-a5d8-634660d84350 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:30.565074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387f2ca-7172-428b-a5d8-634660d84350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:30.586400 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-7387f2ca-7172-428b-a5d8-634660d84350 None None] MFA Rules not processed for user `3ce3e8c1e4f84274a1f5edb940ea4454`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:30.588304 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7387f2ca-7172-428b-a5d8-634660d84350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:30.596008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1356/2711] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:33.008029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee72009-c23a-42be-a938-0ee6c6349be4 service nova] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:33.010810 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2ee72009-c23a-42be-a938-0ee6c6349be4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:33.032975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee72009-c23a-42be-a938-0ee6c6349be4 service nova] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:33.038233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:33.038826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:33.039117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:33.039472 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] PATH_INFO: `/v3/users/a2d7269796cc45a58a01565289763e9c` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:33.042023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d7269796cc45a58a01565289763e9c)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:33.043185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:33.363373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:33.365761 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d7269796cc45a58a01565289763e9c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:33.366465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d7269796cc45a58a01565289763e9c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:33.367057 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d7269796cc45a58a01565289763e9c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:33.375299 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d7269796cc45a58a01565289763e9c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:33.391708 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d7269796cc45a58a01565289763e9c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:33.466029 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1356/2712] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:38:33 2022] DELETE /identity/v3/users/a2d7269796cc45a58a01565289763e9c => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:38:33.467977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbbc66-b756-48ef-8f01-77a6e75daa70 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:33.468495 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbbc66-b756-48ef-8f01-77a6e75daa70 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:33.468592 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbbc66-b756-48ef-8f01-77a6e75daa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:33.489803 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-19cbbc66-b756-48ef-8f01-77a6e75daa70 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:33.491143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-19cbbc66-b756-48ef-8f01-77a6e75daa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:33.557560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1357/2713] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:38:33.569049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:33.570854 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:33.647092 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f535d36-931d-4c55-a4dd-4bb7c914e0ee admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:33.654068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e1195-a500-4b3c-a6f1-bcf9d7197d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:33.655160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e1195-a500-4b3c-a6f1-bcf9d7197d4b service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:33.655835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e1195-a500-4b3c-a6f1-bcf9d7197d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:33.656323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e1195-a500-4b3c-a6f1-bcf9d7197d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:33.673644 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e1195-a500-4b3c-a6f1-bcf9d7197d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:33.679250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e1195-a500-4b3c-a6f1-bcf9d7197d4b service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:33.695068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1357/2714] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:38:34.254868 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbbc66-b756-48ef-8f01-77a6e75daa70 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:34.257780 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-19cbbc66-b756-48ef-8f01-77a6e75daa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:34.272293 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:34.290057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbbc66-b756-48ef-8f01-77a6e75daa70 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:34.298980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:34.300002 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:34.300506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:34.301007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] PATH_INFO: `/v3/projects/f0cf90d3ed5d447fb2f29e745d6435ac` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:34.303000 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0cf90d3ed5d447fb2f29e745d6435ac)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:34.307458 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:34.476921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0cf90d3ed5d447fb2f29e745d6435ac', 'request_id': 'req-1ec11846-5d37-4a41-ac01-e76efcce3b64'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:34.483778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0cf90d3ed5d447fb2f29e745d6435ac', 'request_id': 'req-1ec11846-5d37-4a41-ac01-e76efcce3b64'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:34.927368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0cf90d3ed5d447fb2f29e745d6435ac', 'request_id': 'req-1ec11846-5d37-4a41-ac01-e76efcce3b64'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:34.941173 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] The token cache is being invalidate because project f0cf90d3ed5d447fb2f29e745d6435ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:38:34.941471 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:34.942244 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1ec11846-5d37-4a41-ac01-e76efcce3b64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:34.946641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1358/2715] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:38:34 2022] DELETE /identity/v3/projects/f0cf90d3ed5d447fb2f29e745d6435ac => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:34.952071 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dffb6-672e-45ce-84d2-035fd7cb8f58 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:34.952612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dffb6-672e-45ce-84d2-035fd7cb8f58 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:34.952713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dffb6-672e-45ce-84d2-035fd7cb8f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:34.975192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c67dffb6-672e-45ce-84d2-035fd7cb8f58 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:34.981482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c67dffb6-672e-45ce-84d2-035fd7cb8f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:35.050749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1358/2716] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:35.061701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff837da-7ff7-4560-b060-ac3d10a55da6 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:35.062494 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff837da-7ff7-4560-b060-ac3d10a55da6 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:35.062687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff837da-7ff7-4560-b060-ac3d10a55da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:35.085320 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-6ff837da-7ff7-4560-b060-ac3d10a55da6 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:35.086613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6ff837da-7ff7-4560-b060-ac3d10a55da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:35.102347 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1359/2717] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:35.109291 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dffb6-672e-45ce-84d2-035fd7cb8f58 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:35.122267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dffb6-672e-45ce-84d2-035fd7cb8f58 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:35.128025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e504d4-5b82-4066-b922-f7baf9d791ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:35.128657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e504d4-5b82-4066-b922-f7baf9d791ac admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:35.128941 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e504d4-5b82-4066-b922-f7baf9d791ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:35.130025 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e504d4-5b82-4066-b922-f7baf9d791ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:35.130253 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e504d4-5b82-4066-b922-f7baf9d791ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:38:35.130710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e504d4-5b82-4066-b922-f7baf9d791ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:35.132478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e504d4-5b82-4066-b922-f7baf9d791ac admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:35.147585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1359/2718] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:38:35.151161 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff837da-7ff7-4560-b060-ac3d10a55da6 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:35.160827 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff837da-7ff7-4560-b060-ac3d10a55da6 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:35.166558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664261e-1e37-476a-a4dd-d80550f93c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:35.167195 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664261e-1e37-476a-a4dd-d80550f93c4b admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:35.167489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664261e-1e37-476a-a4dd-d80550f93c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:35.167816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664261e-1e37-476a-a4dd-d80550f93c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:35.168510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664261e-1e37-476a-a4dd-d80550f93c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:35.171275 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664261e-1e37-476a-a4dd-d80550f93c4b admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:35.288666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1360/2719] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:38:35 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:38:35.294954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e504d4-5b82-4066-b922-f7baf9d791ac admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:35.307303 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e504d4-5b82-4066-b922-f7baf9d791ac admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:35.397384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:35.410893 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbd8ed-fa62-43a8-ad6f-ac5e9df96e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:35.411850 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbd8ed-fa62-43a8-ad6f-ac5e9df96e72 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:35.412037 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbd8ed-fa62-43a8-ad6f-ac5e9df96e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:35.412349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbd8ed-fa62-43a8-ad6f-ac5e9df96e72 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:35.413127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbd8ed-fa62-43a8-ad6f-ac5e9df96e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:35.415354 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbd8ed-fa62-43a8-ad6f-ac5e9df96e72 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:35.631654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1360/2720] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:38:35 2022] POST /identity/v3/users => generated 287 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:38:35.636194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664261e-1e37-476a-a4dd-d80550f93c4b admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:35.647442 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664261e-1e37-476a-a4dd-d80550f93c4b admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:35.652976 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb8cd7-dfcc-487f-9216-bc0345445af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:35.653527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb8cd7-dfcc-487f-9216-bc0345445af8 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:35.653810 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb8cd7-dfcc-487f-9216-bc0345445af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:35.654130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb8cd7-dfcc-487f-9216-bc0345445af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:35.654745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb8cd7-dfcc-487f-9216-bc0345445af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:35.656392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb8cd7-dfcc-487f-9216-bc0345445af8 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:35.668900 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1361/2721] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:38:35.673683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbd8ed-fa62-43a8-ad6f-ac5e9df96e72 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:35.687276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbd8ed-fa62-43a8-ad6f-ac5e9df96e72 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:35.694757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d572c-44f9-402b-978b-7fdcde015387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:35.695806 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d572c-44f9-402b-978b-7fdcde015387 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:35.696301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d572c-44f9-402b-978b-7fdcde015387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:35.697451 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d572c-44f9-402b-978b-7fdcde015387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:35.698792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d572c-44f9-402b-978b-7fdcde015387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:35.701276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d572c-44f9-402b-978b-7fdcde015387 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:35.720455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1361/2722] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:38:35.723578 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb8cd7-dfcc-487f-9216-bc0345445af8 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:35.737326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb8cd7-dfcc-487f-9216-bc0345445af8 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:35.744768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a38f1d-c8ff-48a0-b0a3-f3e854d0f217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:35.745629 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a38f1d-c8ff-48a0-b0a3-f3e854d0f217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:35.745981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a38f1d-c8ff-48a0-b0a3-f3e854d0f217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:35.746343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a38f1d-c8ff-48a0-b0a3-f3e854d0f217 admin admin] PATH_INFO: `/v3/projects/60b6671664644e78aba7a09ef8a4d49e/users/cce379f9a4ef4f99ae5052e3d468263c/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:35.769210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a38f1d-c8ff-48a0-b0a3-f3e854d0f217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b6671664644e78aba7a09ef8a4d49e, user_id=cce379f9a4ef4f99ae5052e3d468263c, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:35.774309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a38f1d-c8ff-48a0-b0a3-f3e854d0f217 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:35.939338 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1362/2723] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:38:35 2022] PUT /identity/v3/projects/60b6671664644e78aba7a09ef8a4d49e/users/cce379f9a4ef4f99ae5052e3d468263c/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:35.950247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161db595-859b-4008-a9fe-3e3bc5d87c5d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:35.950696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161db595-859b-4008-a9fe-3e3bc5d87c5d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:35.950948 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161db595-859b-4008-a9fe-3e3bc5d87c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:35.986315 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-161db595-859b-4008-a9fe-3e3bc5d87c5d None None] MFA Rules not processed for user `cce379f9a4ef4f99ae5052e3d468263c`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:35.987589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-161db595-859b-4008-a9fe-3e3bc5d87c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:36.097059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1362/2724] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:38:36.109452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a38f1d-c8ff-48a0-b0a3-f3e854d0f217 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:36.111804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-99a38f1d-c8ff-48a0-b0a3-f3e854d0f217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:36.184576 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a38f1d-c8ff-48a0-b0a3-f3e854d0f217 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:36.189366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f0623-44b6-4043-b148-617b6cc5e23e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:36.194224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f0623-44b6-4043-b148-617b6cc5e23e service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:36.194224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f0623-44b6-4043-b148-617b6cc5e23e service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:36.194224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f0623-44b6-4043-b148-617b6cc5e23e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:36.201173 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f0623-44b6-4043-b148-617b6cc5e23e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:36.202702 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56f0623-44b6-4043-b148-617b6cc5e23e service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:36.219999 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1363/2725] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:38:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:38:38.375314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161db595-859b-4008-a9fe-3e3bc5d87c5d None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:38.377001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-161db595-859b-4008-a9fe-3e3bc5d87c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:38.479123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161db595-859b-4008-a9fe-3e3bc5d87c5d None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:38.487115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:38.488034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:38.488396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:38.488769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] PATH_INFO: `/v3/users/8e7b9d2e98b44681a091c37e888fa867` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:38.490962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e7b9d2e98b44681a091c37e888fa867)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:38.494864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:38.867835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e7b9d2e98b44681a091c37e888fa867'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:38.868726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e7b9d2e98b44681a091c37e888fa867'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:38.869652 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e7b9d2e98b44681a091c37e888fa867'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:38.880545 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e7b9d2e98b44681a091c37e888fa867'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:38.888720 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:38.899263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e7b9d2e98b44681a091c37e888fa867'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:39.015138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1363/2726] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:38:38 2022] DELETE /identity/v3/users/8e7b9d2e98b44681a091c37e888fa867 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:39.020344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c9ab4-8889-4563-8176-730e27cb0d32 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:39.020919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c9ab4-8889-4563-8176-730e27cb0d32 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:39.020919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c9ab4-8889-4563-8176-730e27cb0d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:39.041919 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-cd5c9ab4-8889-4563-8176-730e27cb0d32 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:39.043130 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cd5c9ab4-8889-4563-8176-730e27cb0d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:39.067317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:39.131343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1364/2727] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:39.142691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:39.144633 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:39.270185 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd068d-c1fd-4b00-a51d-ce3a3bae4138 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:39.274670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e0927-30f3-4610-9f4c-859d578492a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:39.275316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0927-30f3-4610-9f4c-859d578492a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:39.275646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0927-30f3-4610-9f4c-859d578492a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:39.275990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e0927-30f3-4610-9f4c-859d578492a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:39.297068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e0927-30f3-4610-9f4c-859d578492a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:39.299465 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1e0927-30f3-4610-9f4c-859d578492a5 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:39.312927 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1364/2728] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:38:39.831350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c9ab4-8889-4563-8176-730e27cb0d32 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:39.833250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cd5c9ab4-8889-4563-8176-730e27cb0d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:39.859327 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c9ab4-8889-4563-8176-730e27cb0d32 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:39.882841 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:39.884471 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:39.884864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:39.885152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] PATH_INFO: `/v3/projects/67af3d29165e46288763e578d87ffbb2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:39.886422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67af3d29165e46288763e578d87ffbb2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:39.888437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:40.052684 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67af3d29165e46288763e578d87ffbb2', 'request_id': 'req-2775855e-3b3a-4e11-adc4-cc645ab57560'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:40.063105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67af3d29165e46288763e578d87ffbb2', 'request_id': 'req-2775855e-3b3a-4e11-adc4-cc645ab57560'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:40.184923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67af3d29165e46288763e578d87ffbb2', 'request_id': 'req-2775855e-3b3a-4e11-adc4-cc645ab57560'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:40.200030 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] The token cache is being invalidate because project 67af3d29165e46288763e578d87ffbb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:38:40.200300 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:40.201009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-2775855e-3b3a-4e11-adc4-cc645ab57560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:40.202555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1365/2729] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:38:39 2022] DELETE /identity/v3/projects/67af3d29165e46288763e578d87ffbb2 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:40.209434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32ec02-0d6e-4e4f-b65b-8dfea5040fd1 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:40.210009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32ec02-0d6e-4e4f-b65b-8dfea5040fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:40.210009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a32ec02-0d6e-4e4f-b65b-8dfea5040fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:40.232444 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6a32ec02-0d6e-4e4f-b65b-8dfea5040fd1 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:40.233697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6a32ec02-0d6e-4e4f-b65b-8dfea5040fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:40.315200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1365/2730] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:40.323344 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe2499-26ac-492b-8d00-e7430ba1902d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:40.323661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe2499-26ac-492b-8d00-e7430ba1902d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:40.323932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe2499-26ac-492b-8d00-e7430ba1902d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:40.346912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-a8fe2499-26ac-492b-8d00-e7430ba1902d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:40.348546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a8fe2499-26ac-492b-8d00-e7430ba1902d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:40.357701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1366/2731] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:40.362431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32ec02-0d6e-4e4f-b65b-8dfea5040fd1 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:40.380818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a32ec02-0d6e-4e4f-b65b-8dfea5040fd1 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:40.387403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153a2f7-803d-47e3-a6a8-953b67b873a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:40.388330 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153a2f7-803d-47e3-a6a8-953b67b873a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:40.388572 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153a2f7-803d-47e3-a6a8-953b67b873a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:40.388853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153a2f7-803d-47e3-a6a8-953b67b873a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:40.389528 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153a2f7-803d-47e3-a6a8-953b67b873a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:38:40.389835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153a2f7-803d-47e3-a6a8-953b67b873a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:40.391504 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0153a2f7-803d-47e3-a6a8-953b67b873a9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:40.410036 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1366/2732] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:38:40.412350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe2499-26ac-492b-8d00-e7430ba1902d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:40.430339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe2499-26ac-492b-8d00-e7430ba1902d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:40.438942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075e5f3-4e64-4663-9bd4-f41b97fa94cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:40.440094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e5f3-4e64-4663-9bd4-f41b97fa94cc admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:40.440555 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e5f3-4e64-4663-9bd4-f41b97fa94cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:40.441055 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e5f3-4e64-4663-9bd4-f41b97fa94cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:40.442230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075e5f3-4e64-4663-9bd4-f41b97fa94cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:40.446090 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075e5f3-4e64-4663-9bd4-f41b97fa94cc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:40.575078 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1367/2733] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:38:40 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:38:40.580027 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153a2f7-803d-47e3-a6a8-953b67b873a9 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:40.591181 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0153a2f7-803d-47e3-a6a8-953b67b873a9 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:40.656364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae84f22-dee3-4166-9542-ee2fb5e81bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:40.657166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae84f22-dee3-4166-9542-ee2fb5e81bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:40.657631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae84f22-dee3-4166-9542-ee2fb5e81bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:40.658077 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae84f22-dee3-4166-9542-ee2fb5e81bb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:40.658833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae84f22-dee3-4166-9542-ee2fb5e81bb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:40.661151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae84f22-dee3-4166-9542-ee2fb5e81bb2 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:40.784541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:40.791606 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1367/2734] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:38:40 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:38:40.796224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075e5f3-4e64-4663-9bd4-f41b97fa94cc admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:40.807914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075e5f3-4e64-4663-9bd4-f41b97fa94cc admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:40.813355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bf344-d2cc-43ae-9e5e-2faf4de43fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:40.813979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bf344-d2cc-43ae-9e5e-2faf4de43fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:40.814256 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bf344-d2cc-43ae-9e5e-2faf4de43fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:40.814527 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bf344-d2cc-43ae-9e5e-2faf4de43fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:40.815179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bf344-d2cc-43ae-9e5e-2faf4de43fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:40.816933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bf344-d2cc-43ae-9e5e-2faf4de43fa0 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:40.830072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1368/2735] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:38:40.834022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae84f22-dee3-4166-9542-ee2fb5e81bb2 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:40.846814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae84f22-dee3-4166-9542-ee2fb5e81bb2 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:40.852723 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e76edf4-16dd-4248-8529-62043c4c7048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:40.853499 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76edf4-16dd-4248-8529-62043c4c7048 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:40.853851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76edf4-16dd-4248-8529-62043c4c7048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:40.854270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e76edf4-16dd-4248-8529-62043c4c7048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:40.854992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76edf4-16dd-4248-8529-62043c4c7048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:40.857480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e76edf4-16dd-4248-8529-62043c4c7048 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:40.874890 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1368/2736] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:38:40.880605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bf344-d2cc-43ae-9e5e-2faf4de43fa0 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:40.894955 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bf344-d2cc-43ae-9e5e-2faf4de43fa0 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:40.902530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df18ae-0753-4f22-8e5b-bd64015f95d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:40.903899 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df18ae-0753-4f22-8e5b-bd64015f95d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:40.904413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df18ae-0753-4f22-8e5b-bd64015f95d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:40.904932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df18ae-0753-4f22-8e5b-bd64015f95d9 admin admin] PATH_INFO: `/v3/projects/5da3daaaad98431d91c50164a5d864fc/users/a927559e0d9d41e7a48ff8f33ea49f69/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:40.928409 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df18ae-0753-4f22-8e5b-bd64015f95d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5da3daaaad98431d91c50164a5d864fc, user_id=a927559e0d9d41e7a48ff8f33ea49f69, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:40.931964 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df18ae-0753-4f22-8e5b-bd64015f95d9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:41.030154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1369/2737] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:38:40 2022] PUT /identity/v3/projects/5da3daaaad98431d91c50164a5d864fc/users/a927559e0d9d41e7a48ff8f33ea49f69/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:41.043133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddca9fb-8f33-4af4-9028-91d81ce56bc6 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:41.043133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddca9fb-8f33-4af4-9028-91d81ce56bc6 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:41.043133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddca9fb-8f33-4af4-9028-91d81ce56bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:41.072630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-2ddca9fb-8f33-4af4-9028-91d81ce56bc6 None None] MFA Rules not processed for user `a927559e0d9d41e7a48ff8f33ea49f69`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:41.073878 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2ddca9fb-8f33-4af4-9028-91d81ce56bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:41.142975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1369/2738] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:41.154764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df18ae-0753-4f22-8e5b-bd64015f95d9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:41.156634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-95df18ae-0753-4f22-8e5b-bd64015f95d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:41.210157 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:41.223512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df18ae-0753-4f22-8e5b-bd64015f95d9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:41.227291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b9183-f96e-4c16-a3d1-1fe9c938e58a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:41.229396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b9183-f96e-4c16-a3d1-1fe9c938e58a service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:41.229678 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b9183-f96e-4c16-a3d1-1fe9c938e58a service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:41.229979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b9183-f96e-4c16-a3d1-1fe9c938e58a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:41.239190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b9183-f96e-4c16-a3d1-1fe9c938e58a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:41.240608 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b9183-f96e-4c16-a3d1-1fe9c938e58a service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:41.251127 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1370/2739] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:38:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:38:41.468697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddca9fb-8f33-4af4-9028-91d81ce56bc6 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:41.468697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2ddca9fb-8f33-4af4-9028-91d81ce56bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:41.541492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddca9fb-8f33-4af4-9028-91d81ce56bc6 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:41.546535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:41.547259 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:41.547670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:41.547988 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] PATH_INFO: `/v3/users/cce379f9a4ef4f99ae5052e3d468263c` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:41.549210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cce379f9a4ef4f99ae5052e3d468263c)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:41.551470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:41.891688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cce379f9a4ef4f99ae5052e3d468263c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:41.892616 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cce379f9a4ef4f99ae5052e3d468263c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:41.893233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cce379f9a4ef4f99ae5052e3d468263c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:41.901763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cce379f9a4ef4f99ae5052e3d468263c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:41.915543 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cce379f9a4ef4f99ae5052e3d468263c'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:41.919666 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:42.116792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1370/2740] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:38:41 2022] DELETE /identity/v3/users/cce379f9a4ef4f99ae5052e3d468263c => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:42.121426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51681a-34d7-42f2-bb68-a5659d87b715 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:42.121716 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51681a-34d7-42f2-bb68-a5659d87b715 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:42.121993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51681a-34d7-42f2-bb68-a5659d87b715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:42.142670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-df51681a-34d7-42f2-bb68-a5659d87b715 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:42.143928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-df51681a-34d7-42f2-bb68-a5659d87b715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:42.219659 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1371/2741] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:42.239243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:42.239243 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:42.297938 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf0f74-6518-4c9e-89cb-f78eb66674a5 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:42.301661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689ab10-dc81-45d4-9151-ef68087e07df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:42.302320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689ab10-dc81-45d4-9151-ef68087e07df service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:42.302600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689ab10-dc81-45d4-9151-ef68087e07df service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:42.302874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689ab10-dc81-45d4-9151-ef68087e07df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:42.313678 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689ab10-dc81-45d4-9151-ef68087e07df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:42.315182 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689ab10-dc81-45d4-9151-ef68087e07df service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:42.326341 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1371/2742] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:38:42.895211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51681a-34d7-42f2-bb68-a5659d87b715 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:42.897081 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-df51681a-34d7-42f2-bb68-a5659d87b715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:42.911144 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51681a-34d7-42f2-bb68-a5659d87b715 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:42.917097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:42.917751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:42.918031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:42.918306 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] PATH_INFO: `/v3/projects/60b6671664644e78aba7a09ef8a4d49e` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:42.919840 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b6671664644e78aba7a09ef8a4d49e)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:42.921931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:43.116095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b6671664644e78aba7a09ef8a4d49e', 'request_id': 'req-627e6aeb-9951-488d-9129-805d034affa3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:43.123746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b6671664644e78aba7a09ef8a4d49e', 'request_id': 'req-627e6aeb-9951-488d-9129-805d034affa3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:43.287613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b6671664644e78aba7a09ef8a4d49e', 'request_id': 'req-627e6aeb-9951-488d-9129-805d034affa3'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:43.304641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] The token cache is being invalidate because project 60b6671664644e78aba7a09ef8a4d49e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:38:43.304939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:43.306319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-627e6aeb-9951-488d-9129-805d034affa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:43.308443 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1372/2743] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:38:42 2022] DELETE /identity/v3/projects/60b6671664644e78aba7a09ef8a4d49e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:44.882484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689ab10-dc81-45d4-9151-ef68087e07df service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:44.885413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f689ab10-dc81-45d4-9151-ef68087e07df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:44.901289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689ab10-dc81-45d4-9151-ef68087e07df service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:44.968799 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:44.969449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:44.969733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:44.970009 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] PATH_INFO: `/v3/users/a927559e0d9d41e7a48ff8f33ea49f69` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:44.971235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a927559e0d9d41e7a48ff8f33ea49f69)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:44.973487 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:45.149740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a927559e0d9d41e7a48ff8f33ea49f69'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:45.150507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a927559e0d9d41e7a48ff8f33ea49f69'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:45.151221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a927559e0d9d41e7a48ff8f33ea49f69'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:45.161287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a927559e0d9d41e7a48ff8f33ea49f69'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:45.183376 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a927559e0d9d41e7a48ff8f33ea49f69'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:45.276989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1372/2744] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:38:44 2022] DELETE /identity/v3/users/a927559e0d9d41e7a48ff8f33ea49f69 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:45.282404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f7333-08fd-43d8-b18c-14a300294150 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:45.282900 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f7333-08fd-43d8-b18c-14a300294150 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:45.283017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f7333-08fd-43d8-b18c-14a300294150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:45.304363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-457f7333-08fd-43d8-b18c-14a300294150 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:45.305644 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-457f7333-08fd-43d8-b18c-14a300294150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:45.369580 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1373/2745] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:45.380022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:45.381522 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:45.404866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:45.446161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11f810-9614-4360-9360-4e2850457534 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:45.453840 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3dc32-55fc-4104-8f64-f658aef6e373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:45.454488 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3dc32-55fc-4104-8f64-f658aef6e373 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:45.454775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3dc32-55fc-4104-8f64-f658aef6e373 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:45.455078 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3dc32-55fc-4104-8f64-f658aef6e373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:45.466034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3dc32-55fc-4104-8f64-f658aef6e373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:45.467480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e3dc32-55fc-4104-8f64-f658aef6e373 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:45.479566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1373/2746] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:38:45.907035 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f7333-08fd-43d8-b18c-14a300294150 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:45.911933 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-457f7333-08fd-43d8-b18c-14a300294150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:45.922250 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f7333-08fd-43d8-b18c-14a300294150 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:45.927033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:45.927719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:45.927998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:45.928270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] PATH_INFO: `/v3/projects/5da3daaaad98431d91c50164a5d864fc` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:45.929404 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5da3daaaad98431d91c50164a5d864fc)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:45.932222 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:45.940975 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:46.060979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5da3daaaad98431d91c50164a5d864fc', 'request_id': 'req-f53c3bf6-831a-4a97-9888-98ad2f632ca9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:46.066746 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5da3daaaad98431d91c50164a5d864fc', 'request_id': 'req-f53c3bf6-831a-4a97-9888-98ad2f632ca9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:46.138328 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5da3daaaad98431d91c50164a5d864fc', 'request_id': 'req-f53c3bf6-831a-4a97-9888-98ad2f632ca9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:46.150721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] The token cache is being invalidate because project 5da3daaaad98431d91c50164a5d864fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:38:46.151032 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:46.151796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-f53c3bf6-831a-4a97-9888-98ad2f632ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:38:46.153723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1374/2747] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:38:45 2022] DELETE /identity/v3/projects/5da3daaaad98431d91c50164a5d864fc => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:46.159101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a63da-7da0-401d-9a31-ab03af3b53e8 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:46.159637 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a63da-7da0-401d-9a31-ab03af3b53e8 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:46.159742 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a63da-7da0-401d-9a31-ab03af3b53e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:46.181018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-1c7a63da-7da0-401d-9a31-ab03af3b53e8 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:46.182131 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1c7a63da-7da0-401d-9a31-ab03af3b53e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:46.257922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1374/2748] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:46.268426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc4e01-9126-41e8-ab5b-2f619c491ccd None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:46.268862 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc4e01-9126-41e8-ab5b-2f619c491ccd None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:46.269291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc4e01-9126-41e8-ab5b-2f619c491ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:46.294873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-85dc4e01-9126-41e8-ab5b-2f619c491ccd None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:46.304102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-85dc4e01-9126-41e8-ab5b-2f619c491ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:46.304701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1375/2749] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:46.309853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a63da-7da0-401d-9a31-ab03af3b53e8 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:46.321903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a63da-7da0-401d-9a31-ab03af3b53e8 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:46.329670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c177ac-d71d-4855-9ba5-0d1e812128ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:46.330290 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c177ac-d71d-4855-9ba5-0d1e812128ee admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:46.330581 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c177ac-d71d-4855-9ba5-0d1e812128ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:46.330839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c177ac-d71d-4855-9ba5-0d1e812128ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:46.331493 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c177ac-d71d-4855-9ba5-0d1e812128ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:38:46.331836 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c177ac-d71d-4855-9ba5-0d1e812128ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:46.333523 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c177ac-d71d-4855-9ba5-0d1e812128ee admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:46.346281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1375/2750] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:38:46.353734 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc4e01-9126-41e8-ab5b-2f619c491ccd None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:46.363489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dc4e01-9126-41e8-ab5b-2f619c491ccd None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:46.369007 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdc98a-0ed8-47c0-be5f-733c4a951471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:46.369604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdc98a-0ed8-47c0-be5f-733c4a951471 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:46.369881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdc98a-0ed8-47c0-be5f-733c4a951471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:46.370148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcdc98a-0ed8-47c0-be5f-733c4a951471 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:46.370838 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdc98a-0ed8-47c0-be5f-733c4a951471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:46.372875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcdc98a-0ed8-47c0-be5f-733c4a951471 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:46.523152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1376/2751] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:38:46 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:38:46.527318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c177ac-d71d-4855-9ba5-0d1e812128ee admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:46.537648 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c177ac-d71d-4855-9ba5-0d1e812128ee admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:46.603962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1493d-0b8f-4843-8e27-4656e57d9ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:46.604555 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1493d-0b8f-4843-8e27-4656e57d9ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:46.604834 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1493d-0b8f-4843-8e27-4656e57d9ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:46.605107 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d1493d-0b8f-4843-8e27-4656e57d9ce8 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:46.605759 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1493d-0b8f-4843-8e27-4656e57d9ce8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:46.607715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d1493d-0b8f-4843-8e27-4656e57d9ce8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:46.731817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1376/2752] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:38:46 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:38:46.733526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdc98a-0ed8-47c0-be5f-733c4a951471 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:46.744282 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcdc98a-0ed8-47c0-be5f-733c4a951471 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:46.749008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2741f-c196-414c-866a-16c79963f38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:46.749617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2741f-c196-414c-866a-16c79963f38b admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:46.749894 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2741f-c196-414c-866a-16c79963f38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:46.750164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2741f-c196-414c-866a-16c79963f38b admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:46.750816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2741f-c196-414c-866a-16c79963f38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:46.752451 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2741f-c196-414c-866a-16c79963f38b admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:46.764419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1377/2753] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:38:46.768301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1493d-0b8f-4843-8e27-4656e57d9ce8 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:46.779265 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d1493d-0b8f-4843-8e27-4656e57d9ce8 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:46.784152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68eaf7-10ea-4e41-83a7-2628c49eec80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:46.784762 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68eaf7-10ea-4e41-83a7-2628c49eec80 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:46.785043 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68eaf7-10ea-4e41-83a7-2628c49eec80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:46.785312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68eaf7-10ea-4e41-83a7-2628c49eec80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:46.785960 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68eaf7-10ea-4e41-83a7-2628c49eec80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:46.787542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68eaf7-10ea-4e41-83a7-2628c49eec80 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:46.799862 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1377/2754] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:38:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:38:46.807921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2741f-c196-414c-866a-16c79963f38b admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:46.823083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2741f-c196-414c-866a-16c79963f38b admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:46.827934 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fbb88-8f6d-4127-bc74-8a98f2ee6d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:46.828586 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fbb88-8f6d-4127-bc74-8a98f2ee6d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:46.828873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fbb88-8f6d-4127-bc74-8a98f2ee6d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:46.829146 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fbb88-8f6d-4127-bc74-8a98f2ee6d7b admin admin] PATH_INFO: `/v3/projects/f5c5ef59b1b44d8a916be97d8d11493f/users/ad7c407cfe924b7a8652f8387c7f77b4/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:46.845562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fbb88-8f6d-4127-bc74-8a98f2ee6d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c5ef59b1b44d8a916be97d8d11493f, user_id=ad7c407cfe924b7a8652f8387c7f77b4, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:46.848550 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fbb88-8f6d-4127-bc74-8a98f2ee6d7b admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:46.996818 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1378/2755] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:38:46 2022] PUT /identity/v3/projects/f5c5ef59b1b44d8a916be97d8d11493f/users/ad7c407cfe924b7a8652f8387c7f77b4/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:47.002399 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501fa29-870c-49e2-a233-13e5151d43d9 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:47.002713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501fa29-870c-49e2-a233-13e5151d43d9 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:47.002958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4501fa29-870c-49e2-a233-13e5151d43d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:47.022915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-4501fa29-870c-49e2-a233-13e5151d43d9 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:47.024270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4501fa29-870c-49e2-a233-13e5151d43d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:47.052331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:47.098828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1378/2756] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:47.109550 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fbb88-8f6d-4127-bc74-8a98f2ee6d7b admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:47.111380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-931fbb88-8f6d-4127-bc74-8a98f2ee6d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:47.177765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fbb88-8f6d-4127-bc74-8a98f2ee6d7b admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:47.182698 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae39f5-af6c-4cc1-91a4-9936fe6fca1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:47.183463 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae39f5-af6c-4cc1-91a4-9936fe6fca1b service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:47.183901 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae39f5-af6c-4cc1-91a4-9936fe6fca1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:47.184294 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae39f5-af6c-4cc1-91a4-9936fe6fca1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:47.195643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae39f5-af6c-4cc1-91a4-9936fe6fca1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:47.197693 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae39f5-af6c-4cc1-91a4-9936fe6fca1b service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:47.208288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1379/2757] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:38:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:38:53.387191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e15e3-9ddc-4f7d-a538-2c264f780528 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:53.387191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e15e3-9ddc-4f7d-a538-2c264f780528 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:53.387191 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e15e3-9ddc-4f7d-a538-2c264f780528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:53.415835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-866e15e3-9ddc-4f7d-a538-2c264f780528 None None] MFA Rules not processed for user `ad7c407cfe924b7a8652f8387c7f77b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:38:53.417192 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-866e15e3-9ddc-4f7d-a538-2c264f780528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:38:53.504717 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1379/2758] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:38:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:38:53.509738 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae39f5-af6c-4cc1-91a4-9936fe6fca1b service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:53.521258 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae39f5-af6c-4cc1-91a4-9936fe6fca1b service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:53.528389 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22ae4-d73e-44e9-9209-c175ed6919e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:53.529217 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22ae4-d73e-44e9-9209-c175ed6919e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:53.529649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22ae4-d73e-44e9-9209-c175ed6919e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:53.530126 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22ae4-d73e-44e9-9209-c175ed6919e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:53.531331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda22ae4-d73e-44e9-9209-c175ed6919e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:53.535866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda22ae4-d73e-44e9-9209-c175ed6919e1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:53.712270 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1380/2759] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:38:53 2022] POST /identity/v3/projects => generated 379 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:38:53.716535 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e15e3-9ddc-4f7d-a538-2c264f780528 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:53.729058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e15e3-9ddc-4f7d-a538-2c264f780528 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:53.822199 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a716a1-fcce-4c43-aa9c-508145acebe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:53.822911 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a716a1-fcce-4c43-aa9c-508145acebe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:53.823591 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a716a1-fcce-4c43-aa9c-508145acebe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:53.823880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a716a1-fcce-4c43-aa9c-508145acebe1 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:53.824576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a716a1-fcce-4c43-aa9c-508145acebe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:53.826851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a716a1-fcce-4c43-aa9c-508145acebe1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:53.927734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1380/2760] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:38:53 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:38:53.931505 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22ae4-d73e-44e9-9209-c175ed6919e1 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:53.943047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22ae4-d73e-44e9-9209-c175ed6919e1 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:53.949045 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3d9d9-1f75-4ce8-b110-72dae05f188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:53.949669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3d9d9-1f75-4ce8-b110-72dae05f188d admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:53.949956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3d9d9-1f75-4ce8-b110-72dae05f188d admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:53.950235 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a3d9d9-1f75-4ce8-b110-72dae05f188d admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:53.950944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3d9d9-1f75-4ce8-b110-72dae05f188d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:53.952631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a3d9d9-1f75-4ce8-b110-72dae05f188d admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:53.959236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:38:53.968517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1381/2761] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:38:53.969845 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a716a1-fcce-4c43-aa9c-508145acebe1 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:53.983218 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a716a1-fcce-4c43-aa9c-508145acebe1 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:53.988615 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5825992-8688-4b7c-b0d3-3e934b04b4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:53.991837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5825992-8688-4b7c-b0d3-3e934b04b4a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:53.991837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5825992-8688-4b7c-b0d3-3e934b04b4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:53.991837 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5825992-8688-4b7c-b0d3-3e934b04b4a3 admin admin] PATH_INFO: `/v3/projects/6f2ee0c5460b4ca0ad29c419dab374a0/users/33188bf83b6845faa94668978f911a5e/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:54.010798 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5825992-8688-4b7c-b0d3-3e934b04b4a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2ee0c5460b4ca0ad29c419dab374a0, user_id=33188bf83b6845faa94668978f911a5e, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:54.013719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5825992-8688-4b7c-b0d3-3e934b04b4a3 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:54.110344 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1381/2762] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:38:53 2022] PUT /identity/v3/projects/6f2ee0c5460b4ca0ad29c419dab374a0/users/33188bf83b6845faa94668978f911a5e/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:38:54.114380 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3d9d9-1f75-4ce8-b110-72dae05f188d admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:54.125674 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a3d9d9-1f75-4ce8-b110-72dae05f188d admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:54.186551 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d02c7b-de33-40e1-b898-f194a728f168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:54.187110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02c7b-de33-40e1-b898-f194a728f168 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:54.187392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02c7b-de33-40e1-b898-f194a728f168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:54.187709 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d02c7b-de33-40e1-b898-f194a728f168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:54.188414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d02c7b-de33-40e1-b898-f194a728f168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:54.190051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d02c7b-de33-40e1-b898-f194a728f168 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:54.201455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1382/2763] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:38:54.205857 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5825992-8688-4b7c-b0d3-3e934b04b4a3 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:38:54.216475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5825992-8688-4b7c-b0d3-3e934b04b4a3 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:38:54.221238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c50b5e-3f9c-409a-8e70-5f850b48d111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:38:54.221895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c50b5e-3f9c-409a-8e70-5f850b48d111 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:38:54.222180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c50b5e-3f9c-409a-8e70-5f850b48d111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:38:54.222695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c50b5e-3f9c-409a-8e70-5f850b48d111 admin admin] PATH_INFO: `/v3/projects/6f2ee0c5460b4ca0ad29c419dab374a0/users/33188bf83b6845faa94668978f911a5e/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:38:54.224809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c50b5e-3f9c-409a-8e70-5f850b48d111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2ee0c5460b4ca0ad29c419dab374a0, user_id=33188bf83b6845faa94668978f911a5e, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:38:54.227462 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c50b5e-3f9c-409a-8e70-5f850b48d111 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:38:54.248971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1382/2764] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:38:54 2022] PUT /identity/v3/projects/6f2ee0c5460b4ca0ad29c419dab374a0/users/33188bf83b6845faa94668978f911a5e/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:39:01.298257 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36547464-dfe1-4f6e-a11b-96491fe8968a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:39:01.298822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36547464-dfe1-4f6e-a11b-96491fe8968a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:39:01.299309 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36547464-dfe1-4f6e-a11b-96491fe8968a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:39:01.335318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-36547464-dfe1-4f6e-a11b-96491fe8968a None None] MFA Rules not processed for user `33188bf83b6845faa94668978f911a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:39:01.337274 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-36547464-dfe1-4f6e-a11b-96491fe8968a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:39:01.439105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1383/2765] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:39:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:39:01.456953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c50b5e-3f9c-409a-8e70-5f850b48d111 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:39:01.459124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c6c50b5e-3f9c-409a-8e70-5f850b48d111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:39:01.478756 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:39:01.589383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c50b5e-3f9c-409a-8e70-5f850b48d111 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:39:01.598641 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf6048-8e7f-43c4-b7b3-591e661a0aa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:39:01.600284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf6048-8e7f-43c4-b7b3-591e661a0aa9 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:39:01.600284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf6048-8e7f-43c4-b7b3-591e661a0aa9 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:39:01.600888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf6048-8e7f-43c4-b7b3-591e661a0aa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:39:01.746682 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf6048-8e7f-43c4-b7b3-591e661a0aa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:39:01.750668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caf6048-8e7f-43c4-b7b3-591e661a0aa9 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:39:01.771384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1383/2766] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:39:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:39:11.434676 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=63 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:39:11.761903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=52 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:39:23.028011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499da5e5-2aa7-4a5c-bd20-b0a43fb8dfa4 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:39:23.028908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499da5e5-2aa7-4a5c-bd20-b0a43fb8dfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:39:23.029116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499da5e5-2aa7-4a5c-bd20-b0a43fb8dfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:39:23.049238 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-499da5e5-2aa7-4a5c-bd20-b0a43fb8dfa4 None None] MFA Rules not processed for user `ad7c407cfe924b7a8652f8387c7f77b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:39:23.050490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-499da5e5-2aa7-4a5c-bd20-b0a43fb8dfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:39:23.058155 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1384/2767] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:39:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:39:33.043325 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:39:39.254477 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c5220-d714-4114-99cb-1946ff6d844c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:39:39.255263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c5220-d714-4114-99cb-1946ff6d844c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:39:39.255263 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c5220-d714-4114-99cb-1946ff6d844c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:39:39.277456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f40c5220-d714-4114-99cb-1946ff6d844c None None] MFA Rules not processed for user `3ce3e8c1e4f84274a1f5edb940ea4454`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:39:39.278751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f40c5220-d714-4114-99cb-1946ff6d844c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:39:39.351586 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1384/2768] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:39:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:39:41.438465 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499da5e5-2aa7-4a5c-bd20-b0a43fb8dfa4 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:39:41.440143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-499da5e5-2aa7-4a5c-bd20-b0a43fb8dfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:39:41.497401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c5220-d714-4114-99cb-1946ff6d844c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:39:41.507822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499da5e5-2aa7-4a5c-bd20-b0a43fb8dfa4 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:39:41.509712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c5220-d714-4114-99cb-1946ff6d844c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:39:41.511587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f621d1-f6f5-4e98-aa83-7908a5eff00b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:39:41.512210 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f621d1-f6f5-4e98-aa83-7908a5eff00b service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:39:41.512493 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f621d1-f6f5-4e98-aa83-7908a5eff00b service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:39:41.512811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f621d1-f6f5-4e98-aa83-7908a5eff00b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:39:41.513733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d441e-7a19-448b-9c22-527587588540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:39:41.514498 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d441e-7a19-448b-9c22-527587588540 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:39:41.514614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d8f621d1-f6f5-4e98-aa83-7908a5eff00b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:39:41.514912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d441e-7a19-448b-9c22-527587588540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:39:41.515196 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d441e-7a19-448b-9c22-527587588540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:39:41.516982 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c38d441e-7a19-448b-9c22-527587588540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:39:41.531786 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d441e-7a19-448b-9c22-527587588540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:39:41.533281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d441e-7a19-448b-9c22-527587588540 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:39:41.546387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1385/2769] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:39:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:39:41.642134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f621d1-f6f5-4e98-aa83-7908a5eff00b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:39:41.642134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f621d1-f6f5-4e98-aa83-7908a5eff00b service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:39:41.642134 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1385/2770] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:39:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:39:43.499435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d441e-7a19-448b-9c22-527587588540 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:39:43.510473 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d441e-7a19-448b-9c22-527587588540 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:39:43.516316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42875cfe-aa09-4e5b-8e84-d94f5f1a6e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:39:43.517323 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42875cfe-aa09-4e5b-8e84-d94f5f1a6e8b service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:39:43.517691 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42875cfe-aa09-4e5b-8e84-d94f5f1a6e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:39:43.517899 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42875cfe-aa09-4e5b-8e84-d94f5f1a6e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:39:43.520974 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-42875cfe-aa09-4e5b-8e84-d94f5f1a6e8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:39:43.536584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42875cfe-aa09-4e5b-8e84-d94f5f1a6e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:39:43.538554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42875cfe-aa09-4e5b-8e84-d94f5f1a6e8b service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:39:43.551314 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1386/2771] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:39:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:39:49.405851 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f621d1-f6f5-4e98-aa83-7908a5eff00b service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:39:49.418319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f621d1-f6f5-4e98-aa83-7908a5eff00b service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:39:49.422556 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4f0fb-1812-4ce3-890f-6add38e4beae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:39:49.423184 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4f0fb-1812-4ce3-890f-6add38e4beae service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:39:49.423508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4f0fb-1812-4ce3-890f-6add38e4beae service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:39:49.423815 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4f0fb-1812-4ce3-890f-6add38e4beae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:39:49.425613 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-dfb4f0fb-1812-4ce3-890f-6add38e4beae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:39:49.494428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4f0fb-1812-4ce3-890f-6add38e4beae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:39:49.495906 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb4f0fb-1812-4ce3-890f-6add38e4beae service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:39:49.505923 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1386/2772] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:39:51.882728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42875cfe-aa09-4e5b-8e84-d94f5f1a6e8b service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:39:51.895996 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42875cfe-aa09-4e5b-8e84-d94f5f1a6e8b service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:39:51.903146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdf37a-fc11-457b-ad9a-0b2848faa610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:39:51.904645 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdf37a-fc11-457b-ad9a-0b2848faa610 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:39:51.905114 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdf37a-fc11-457b-ad9a-0b2848faa610 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:39:51.905631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdf37a-fc11-457b-ad9a-0b2848faa610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:39:51.916346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdf37a-fc11-457b-ad9a-0b2848faa610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:39:51.918282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdf37a-fc11-457b-ad9a-0b2848faa610 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:39:51.931842 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1387/2773] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:39:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:39:59.069511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4f0fb-1812-4ce3-890f-6add38e4beae service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:39:59.071985 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-dfb4f0fb-1812-4ce3-890f-6add38e4beae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:39:59.087029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4f0fb-1812-4ce3-890f-6add38e4beae service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:39:59.093493 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:39:59.094158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:39:59.094434 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:39:59.094724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] PATH_INFO: `/v3/users/8089cf57e0014489a12e659278c2931e` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:39:59.096836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8089cf57e0014489a12e659278c2931e)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:39:59.100915 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:39:59.436009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:39:59.441267 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8089cf57e0014489a12e659278c2931e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:39:59.587858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8089cf57e0014489a12e659278c2931e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:39:59.588515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8089cf57e0014489a12e659278c2931e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:39:59.603616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8089cf57e0014489a12e659278c2931e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:39:59.604706 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8089cf57e0014489a12e659278c2931e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:39:59.620749 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1387/2774] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:39:59 2022] DELETE /identity/v3/users/8089cf57e0014489a12e659278c2931e => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:39:59.933221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdf37a-fc11-457b-ad9a-0b2848faa610 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:39:59.944607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdf37a-fc11-457b-ad9a-0b2848faa610 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:40:00.011177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:40:00.011763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:40:00.012048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:40:00.012324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] PATH_INFO: `/v3/projects/600dd39d7a7944a49330866c0e26cb66` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:40:00.013469 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=600dd39d7a7944a49330866c0e26cb66)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:40:00.015472 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:40:00.176229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600dd39d7a7944a49330866c0e26cb66', 'request_id': 'req-21d3e998-2f3e-45e9-b161-8497bde49313'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:00.182697 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600dd39d7a7944a49330866c0e26cb66', 'request_id': 'req-21d3e998-2f3e-45e9-b161-8497bde49313'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:00.307769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600dd39d7a7944a49330866c0e26cb66', 'request_id': 'req-21d3e998-2f3e-45e9-b161-8497bde49313'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:00.313166 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:40:00.319817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] The token cache is being invalidate because project 600dd39d7a7944a49330866c0e26cb66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:40:00.320109 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:00.321364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:00.322926 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1388/2775] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:39:59 2022] DELETE /identity/v3/projects/600dd39d7a7944a49330866c0e26cb66 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:40:00.327125 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:40:00.328737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:00.342668 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a158aa7-6512-49f0-a339-03403ca28e08 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:40:00.401087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:40:00.401726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:40:00.402011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:40:00.402289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] PATH_INFO: `/v3/users/c92a1ec9f82a4fd3886b67bf743529cd` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:40:00.403457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92a1ec9f82a4fd3886b67bf743529cd)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:40:00.405419 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:40:00.631280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92a1ec9f82a4fd3886b67bf743529cd'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:00.719290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92a1ec9f82a4fd3886b67bf743529cd'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:00.719981 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92a1ec9f82a4fd3886b67bf743529cd'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:00.734361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92a1ec9f82a4fd3886b67bf743529cd'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:00.735108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92a1ec9f82a4fd3886b67bf743529cd'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:00.749456 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1388/2776] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:40:00 2022] DELETE /identity/v3/users/c92a1ec9f82a4fd3886b67bf743529cd => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:40:01.035768 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:40:01.037461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:01.052816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e998-2f3e-45e9-b161-8497bde49313 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:40:01.116198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:40:01.116852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:40:01.117129 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:40:01.117402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] PATH_INFO: `/v3/projects/81dd2c4296a5457894420a8114b1ab5d` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:40:01.118717 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] RBAC: Authorizing `identity:delete_project(project_id=81dd2c4296a5457894420a8114b1ab5d)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:40:01.121148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:40:01.253466 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81dd2c4296a5457894420a8114b1ab5d', 'request_id': 'req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:01.260480 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81dd2c4296a5457894420a8114b1ab5d', 'request_id': 'req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:01.348119 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81dd2c4296a5457894420a8114b1ab5d', 'request_id': 'req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:01.359954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] The token cache is being invalidate because project 81dd2c4296a5457894420a8114b1ab5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:40:01.360247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:01.361018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8991f2ae-a2e7-47b8-97ec-1ba01bdba99d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:40:01.362602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1389/2777] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:40:01 2022] DELETE /identity/v3/projects/81dd2c4296a5457894420a8114b1ab5d => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:40:10.745723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:40:11.356877 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:40:37.591291 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:40:37.596108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:37.737139 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51ac18-3e7d-46dd-823e-7e99bcd8577f admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:40:37.846837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa671335-79b4-42eb-8888-0c84c75a7f1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:40:37.848116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa671335-79b4-42eb-8888-0c84c75a7f1b service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:40:37.849106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa671335-79b4-42eb-8888-0c84c75a7f1b service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:40:37.849946 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa671335-79b4-42eb-8888-0c84c75a7f1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:40:37.855882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-aa671335-79b4-42eb-8888-0c84c75a7f1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:37.889967 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa671335-79b4-42eb-8888-0c84c75a7f1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:40:37.892621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa671335-79b4-42eb-8888-0c84c75a7f1b service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:40:37.911861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1389/2778] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:40:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:40:47.906025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=62 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:40:52.246663 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2c4e5-c0b0-4bb4-9f4c-3a24890a6434 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:40:52.247280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2c4e5-c0b0-4bb4-9f4c-3a24890a6434 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:40:52.247280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2c4e5-c0b0-4bb4-9f4c-3a24890a6434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:40:52.267026 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-2cb2c4e5-c0b0-4bb4-9f4c-3a24890a6434 None None] MFA Rules not processed for user `3ce3e8c1e4f84274a1f5edb940ea4454`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:40:52.268321 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2cb2c4e5-c0b0-4bb4-9f4c-3a24890a6434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:52.332042 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1390/2779] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:40:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:40:52.668100 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa671335-79b4-42eb-8888-0c84c75a7f1b service nova] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:40:52.669796 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-aa671335-79b4-42eb-8888-0c84c75a7f1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:52.732947 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa671335-79b4-42eb-8888-0c84c75a7f1b service nova] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:40:52.736879 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da66586-ddc2-4e7d-830c-8d92cfeeec1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:40:52.737496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da66586-ddc2-4e7d-830c-8d92cfeeec1d service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:40:52.737784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da66586-ddc2-4e7d-830c-8d92cfeeec1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:40:52.738057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da66586-ddc2-4e7d-830c-8d92cfeeec1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:40:52.740003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8da66586-ddc2-4e7d-830c-8d92cfeeec1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:52.750444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da66586-ddc2-4e7d-830c-8d92cfeeec1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:40:52.752504 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da66586-ddc2-4e7d-830c-8d92cfeeec1d service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:40:52.761154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1390/2780] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:40:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:40:53.191285 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2c4e5-c0b0-4bb4-9f4c-3a24890a6434 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:40:53.192966 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2cb2c4e5-c0b0-4bb4-9f4c-3a24890a6434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:53.234743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da66586-ddc2-4e7d-830c-8d92cfeeec1d service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:40:53.251587 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8da66586-ddc2-4e7d-830c-8d92cfeeec1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:53.259816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da66586-ddc2-4e7d-830c-8d92cfeeec1d service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:40:53.264200 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365a565-2e97-4c62-8dd7-cb4d24cba089 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:40:53.264917 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365a565-2e97-4c62-8dd7-cb4d24cba089 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:40:53.265206 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365a565-2e97-4c62-8dd7-cb4d24cba089 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:40:53.265457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365a565-2e97-4c62-8dd7-cb4d24cba089 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:40:53.267410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e365a565-2e97-4c62-8dd7-cb4d24cba089 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:53.278911 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365a565-2e97-4c62-8dd7-cb4d24cba089 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:40:53.280428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e365a565-2e97-4c62-8dd7-cb4d24cba089 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:40:53.281324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2c4e5-c0b0-4bb4-9f4c-3a24890a6434 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:40:53.286903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a02273-60e1-445a-b418-dcc251acf6b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:40:53.287653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a02273-60e1-445a-b418-dcc251acf6b3 service placement] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:40:53.287953 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a02273-60e1-445a-b418-dcc251acf6b3 service placement] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:40:53.288236 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a02273-60e1-445a-b418-dcc251acf6b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:40:53.290416 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1391/2781] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:40:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:40:53.292221 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-14a02273-60e1-445a-b418-dcc251acf6b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:40:53.306612 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a02273-60e1-445a-b418-dcc251acf6b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:40:53.308320 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a02273-60e1-445a-b418-dcc251acf6b3 service placement] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:40:53.320509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1391/2782] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:40:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:41:03.286874 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=32 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:41:03.315089 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=57 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:41:07.429750 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365a565-2e97-4c62-8dd7-cb4d24cba089 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:41:07.431141 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e365a565-2e97-4c62-8dd7-cb4d24cba089 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:41:07.447322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365a565-2e97-4c62-8dd7-cb4d24cba089 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:41:07.453247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550605b-e33a-4896-b036-bce48b15c27d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:41:07.453869 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550605b-e33a-4896-b036-bce48b15c27d service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:41:07.454142 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550605b-e33a-4896-b036-bce48b15c27d service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:41:07.454415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550605b-e33a-4896-b036-bce48b15c27d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:41:07.457727 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0550605b-e33a-4896-b036-bce48b15c27d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:41:07.471350 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550605b-e33a-4896-b036-bce48b15c27d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:41:07.473625 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550605b-e33a-4896-b036-bce48b15c27d service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:41:07.485577 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1392/2783] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:41:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:41:17.480752 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=6 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:41:57.393962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb315a-6b2a-4bda-8e0b-1b0a69132f8e None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:41:57.394511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb315a-6b2a-4bda-8e0b-1b0a69132f8e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:41:57.394511 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb315a-6b2a-4bda-8e0b-1b0a69132f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:41:57.420082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-cfeb315a-6b2a-4bda-8e0b-1b0a69132f8e None None] MFA Rules not processed for user `ad7c407cfe924b7a8652f8387c7f77b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:41:57.421452 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-cfeb315a-6b2a-4bda-8e0b-1b0a69132f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:41:57.497132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1392/2784] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:41:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:42:06.705292 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283d87b-6fd7-4d2c-9ccb-4bb4547d48f4 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:42:06.706369 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283d87b-6fd7-4d2c-9ccb-4bb4547d48f4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:42:06.706718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283d87b-6fd7-4d2c-9ccb-4bb4547d48f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:42:06.728131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-c283d87b-6fd7-4d2c-9ccb-4bb4547d48f4 None None] MFA Rules not processed for user `3ce3e8c1e4f84274a1f5edb940ea4454`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:42:06.729376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-c283d87b-6fd7-4d2c-9ccb-4bb4547d48f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:42:06.738875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1393/2785] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:42:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:42:07.489226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=25 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:42:13.799992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb315a-6b2a-4bda-8e0b-1b0a69132f8e None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:42:13.813128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb315a-6b2a-4bda-8e0b-1b0a69132f8e None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:42:13.827097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835cdd5-4485-45d0-abfb-0264f5759a07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:42:13.827097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835cdd5-4485-45d0-abfb-0264f5759a07 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:42:13.827097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835cdd5-4485-45d0-abfb-0264f5759a07 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:42:13.827097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835cdd5-4485-45d0-abfb-0264f5759a07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:42:13.827758 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1835cdd5-4485-45d0-abfb-0264f5759a07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:42:13.841148 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835cdd5-4485-45d0-abfb-0264f5759a07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:42:13.842788 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1835cdd5-4485-45d0-abfb-0264f5759a07 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:42:13.855251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1393/2786] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:42:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:42:16.721866 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:42:23.850022 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=6 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:42:59.674133 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283d87b-6fd7-4d2c-9ccb-4bb4547d48f4 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:42:59.689171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283d87b-6fd7-4d2c-9ccb-4bb4547d48f4 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:42:59.695800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f17f54-19e2-493e-a640-cc61ce0dd929 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:42:59.696422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17f54-19e2-493e-a640-cc61ce0dd929 service placement] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:42:59.696700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17f54-19e2-493e-a640-cc61ce0dd929 service placement] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:42:59.696971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17f54-19e2-493e-a640-cc61ce0dd929 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:42:59.699201 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-20f17f54-19e2-493e-a640-cc61ce0dd929 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:42:59.712271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f17f54-19e2-493e-a640-cc61ce0dd929 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:42:59.713870 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f17f54-19e2-493e-a640-cc61ce0dd929 service placement] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:42:59.726736 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1394/2787] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:42:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:43:09.720077 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=6 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:43:23.148235 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835cdd5-4485-45d0-abfb-0264f5759a07 service nova] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:23.158063 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1835cdd5-4485-45d0-abfb-0264f5759a07 service nova] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:23.161992 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857929ad-a65f-49bc-a9f3-136996403804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:23.162618 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857929ad-a65f-49bc-a9f3-136996403804 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:23.162901 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857929ad-a65f-49bc-a9f3-136996403804 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:23.164019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857929ad-a65f-49bc-a9f3-136996403804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:23.165110 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-857929ad-a65f-49bc-a9f3-136996403804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:23.252001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857929ad-a65f-49bc-a9f3-136996403804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:23.255434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857929ad-a65f-49bc-a9f3-136996403804 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:23.274098 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1394/2788] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:43:27.108432 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f17f54-19e2-493e-a640-cc61ce0dd929 service placement] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:27.119589 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f17f54-19e2-493e-a640-cc61ce0dd929 service placement] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:27.125091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd8d31-b806-4e2f-9a14-6af8f8ada20d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:27.125728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd8d31-b806-4e2f-9a14-6af8f8ada20d service placement] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:27.126020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd8d31-b806-4e2f-9a14-6af8f8ada20d service placement] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:27.126295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fd8d31-b806-4e2f-9a14-6af8f8ada20d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:27.128154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-98fd8d31-b806-4e2f-9a14-6af8f8ada20d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:27.140076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd8d31-b806-4e2f-9a14-6af8f8ada20d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:27.141671 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fd8d31-b806-4e2f-9a14-6af8f8ada20d service placement] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:27.152816 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1395/2789] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:43:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:43:27.225658 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857929ad-a65f-49bc-a9f3-136996403804 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:27.230370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-857929ad-a65f-49bc-a9f3-136996403804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:27.248165 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857929ad-a65f-49bc-a9f3-136996403804 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:27.253313 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:27.253940 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:27.254220 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:27.254491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] PATH_INFO: `/v3/users/3ce3e8c1e4f84274a1f5edb940ea4454` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:27.255639 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ce3e8c1e4f84274a1f5edb940ea4454)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:27.257677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:27.440673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ce3e8c1e4f84274a1f5edb940ea4454'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:27.442483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ce3e8c1e4f84274a1f5edb940ea4454'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:27.443494 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ce3e8c1e4f84274a1f5edb940ea4454'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:27.457393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ce3e8c1e4f84274a1f5edb940ea4454'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:27.479804 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-c88673e7-acba-415a-b38e-fcb3d99095f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ce3e8c1e4f84274a1f5edb940ea4454'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:27.554826 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1395/2790] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:43:27 2022] DELETE /identity/v3/users/3ce3e8c1e4f84274a1f5edb940ea4454 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:43:27.841825 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd8d31-b806-4e2f-9a14-6af8f8ada20d service placement] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:27.854881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fd8d31-b806-4e2f-9a14-6af8f8ada20d service placement] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:27.917162 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:27.917809 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:27.918091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:27.918368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] PATH_INFO: `/v3/projects/a15460898c2b4b15bfee3d39a01273b4` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:27.919609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15460898c2b4b15bfee3d39a01273b4)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:27.921597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:28.033776 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15460898c2b4b15bfee3d39a01273b4', 'request_id': 'req-14c8545d-ea64-403d-ae7e-a796c872357c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:28.040647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15460898c2b4b15bfee3d39a01273b4', 'request_id': 'req-14c8545d-ea64-403d-ae7e-a796c872357c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:28.099761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15460898c2b4b15bfee3d39a01273b4', 'request_id': 'req-14c8545d-ea64-403d-ae7e-a796c872357c'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:28.112253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] The token cache is being invalidate because project a15460898c2b4b15bfee3d39a01273b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:43:28.113461 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:28.113980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-14c8545d-ea64-403d-ae7e-a796c872357c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:28.115480 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1396/2791] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:43:27 2022] DELETE /identity/v3/projects/a15460898c2b4b15bfee3d39a01273b4 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:43:28.121080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460a42-37a4-4376-bcf4-d5c3c92bab0a None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:28.121393 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460a42-37a4-4376-bcf4-d5c3c92bab0a None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:28.121671 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac460a42-37a4-4376-bcf4-d5c3c92bab0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:28.141554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-ac460a42-37a4-4376-bcf4-d5c3c92bab0a None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:43:28.142771 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ac460a42-37a4-4376-bcf4-d5c3c92bab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:28.201629 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1396/2792] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:43:28.210429 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97096264-200e-46ee-8158-81dcf1037cf7 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:28.210751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97096264-200e-46ee-8158-81dcf1037cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:28.211037 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97096264-200e-46ee-8158-81dcf1037cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:28.232027 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-97096264-200e-46ee-8158-81dcf1037cf7 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:43:28.233261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-97096264-200e-46ee-8158-81dcf1037cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:28.242115 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1397/2793] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:43:28.246580 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac460a42-37a4-4376-bcf4-d5c3c92bab0a None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:28.256424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac460a42-37a4-4376-bcf4-d5c3c92bab0a None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:28.261156 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b2a0e-8d7a-4c3a-9c0b-34d3b764818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:28.261757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b2a0e-8d7a-4c3a-9c0b-34d3b764818d admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:28.262068 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b2a0e-8d7a-4c3a-9c0b-34d3b764818d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:28.262318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b2a0e-8d7a-4c3a-9c0b-34d3b764818d admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:28.263049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b2a0e-8d7a-4c3a-9c0b-34d3b764818d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:43:28.263360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b2a0e-8d7a-4c3a-9c0b-34d3b764818d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:28.264995 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b2a0e-8d7a-4c3a-9c0b-34d3b764818d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:28.266571 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:43:28.277859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1397/2794] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:43:28.285628 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97096264-200e-46ee-8158-81dcf1037cf7 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:28.295019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97096264-200e-46ee-8158-81dcf1037cf7 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:28.299960 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50555439-815e-4809-b694-c4876e8567fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:28.300558 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50555439-815e-4809-b694-c4876e8567fb admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:28.300837 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50555439-815e-4809-b694-c4876e8567fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:28.301173 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50555439-815e-4809-b694-c4876e8567fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:28.301740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50555439-815e-4809-b694-c4876e8567fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:28.303660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50555439-815e-4809-b694-c4876e8567fb admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:28.443938 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1398/2795] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:43:28 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:43:28.448180 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b2a0e-8d7a-4c3a-9c0b-34d3b764818d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:28.459881 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b2a0e-8d7a-4c3a-9c0b-34d3b764818d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:28.517178 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a5ba6-354d-43e2-bbab-a493b5271883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:28.517802 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a5ba6-354d-43e2-bbab-a493b5271883 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:28.518088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a5ba6-354d-43e2-bbab-a493b5271883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:28.518364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a5ba6-354d-43e2-bbab-a493b5271883 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:28.519019 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a5ba6-354d-43e2-bbab-a493b5271883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:28.521559 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a5ba6-354d-43e2-bbab-a493b5271883 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:28.629435 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1398/2796] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:43:28 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:43:28.633751 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50555439-815e-4809-b694-c4876e8567fb admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:28.644036 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50555439-815e-4809-b694-c4876e8567fb admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:28.649513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452b54c-791a-45cc-a81f-330db1686ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:28.650106 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452b54c-791a-45cc-a81f-330db1686ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:28.650384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452b54c-791a-45cc-a81f-330db1686ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:28.650658 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452b54c-791a-45cc-a81f-330db1686ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:28.651301 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452b54c-791a-45cc-a81f-330db1686ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:28.652963 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452b54c-791a-45cc-a81f-330db1686ba1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:28.663179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1399/2797] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:43:28.667287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a5ba6-354d-43e2-bbab-a493b5271883 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:28.677413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a5ba6-354d-43e2-bbab-a493b5271883 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:28.682251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74439b37-4ec1-4625-95f8-e452f88c132a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:28.682852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74439b37-4ec1-4625-95f8-e452f88c132a admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:28.683135 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74439b37-4ec1-4625-95f8-e452f88c132a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:28.683412 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74439b37-4ec1-4625-95f8-e452f88c132a admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:28.684096 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74439b37-4ec1-4625-95f8-e452f88c132a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:28.685695 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74439b37-4ec1-4625-95f8-e452f88c132a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:28.697122 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1399/2798] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:43:28.701211 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452b54c-791a-45cc-a81f-330db1686ba1 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:28.711860 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452b54c-791a-45cc-a81f-330db1686ba1 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:28.716544 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39706779-5542-4fcd-981c-9a24a9929a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:28.717179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39706779-5542-4fcd-981c-9a24a9929a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:28.717457 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39706779-5542-4fcd-981c-9a24a9929a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:28.717731 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39706779-5542-4fcd-981c-9a24a9929a77 admin admin] PATH_INFO: `/v3/projects/ff9961b5ad944a09bf66715de7c32e95/users/c6698f091c9b4bda859b7bda9e83d4cb/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:28.732273 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39706779-5542-4fcd-981c-9a24a9929a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9961b5ad944a09bf66715de7c32e95, user_id=c6698f091c9b4bda859b7bda9e83d4cb, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:28.734912 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39706779-5542-4fcd-981c-9a24a9929a77 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:28.860352 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1400/2799] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:43:28 2022] PUT /identity/v3/projects/ff9961b5ad944a09bf66715de7c32e95/users/c6698f091c9b4bda859b7bda9e83d4cb/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:43:28.871210 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20f017-8410-41b9-b3cb-09f908b91431 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:28.871505 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20f017-8410-41b9-b3cb-09f908b91431 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:28.871817 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d20f017-8410-41b9-b3cb-09f908b91431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:28.905171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-0d20f017-8410-41b9-b3cb-09f908b91431 None None] MFA Rules not processed for user `c6698f091c9b4bda859b7bda9e83d4cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:43:28.915737 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0d20f017-8410-41b9-b3cb-09f908b91431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:28.978954 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1400/2800] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:43:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:43:28.987812 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39706779-5542-4fcd-981c-9a24a9929a77 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:28.990033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-39706779-5542-4fcd-981c-9a24a9929a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:29.005094 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:43:29.067009 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39706779-5542-4fcd-981c-9a24a9929a77 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:29.072822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb31dc-7dfd-4321-884e-3da3cefbb4be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:29.073437 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb31dc-7dfd-4321-884e-3da3cefbb4be service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:29.073724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb31dc-7dfd-4321-884e-3da3cefbb4be service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:29.073997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb31dc-7dfd-4321-884e-3da3cefbb4be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:29.083621 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb31dc-7dfd-4321-884e-3da3cefbb4be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:29.085237 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb31dc-7dfd-4321-884e-3da3cefbb4be service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:29.095331 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1401/2801] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:43:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:43:32.385971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20f017-8410-41b9-b3cb-09f908b91431 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:32.448408 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d20f017-8410-41b9-b3cb-09f908b91431 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:32.453223 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:32.453870 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:32.454167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:32.454436 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] PATH_INFO: `/v3/users/c6698f091c9b4bda859b7bda9e83d4cb` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:32.459702 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6698f091c9b4bda859b7bda9e83d4cb)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:32.461661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:32.464653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:43:32.680533 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6698f091c9b4bda859b7bda9e83d4cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:32.681216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6698f091c9b4bda859b7bda9e83d4cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:32.681863 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6698f091c9b4bda859b7bda9e83d4cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:32.689155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6698f091c9b4bda859b7bda9e83d4cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:32.701326 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6698f091c9b4bda859b7bda9e83d4cb'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:32.772242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1401/2802] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:43:32 2022] DELETE /identity/v3/users/c6698f091c9b4bda859b7bda9e83d4cb => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:43:32.776631 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb64ad-182f-48d2-b5e4-a1b69c7c622b None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:32.777144 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb64ad-182f-48d2-b5e4-a1b69c7c622b None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:32.777259 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb64ad-182f-48d2-b5e4-a1b69c7c622b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:32.797188 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-f7cb64ad-182f-48d2-b5e4-a1b69c7c622b None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:43:32.798414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f7cb64ad-182f-48d2-b5e4-a1b69c7c622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:32.869056 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1402/2803] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:43:32.879789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:32.881280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:32.943742 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f1ba-40b6-4ad8-86bc-e46c404402f7 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:32.947377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697cc88-2995-47ec-a058-49f35418e931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:32.948020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697cc88-2995-47ec-a058-49f35418e931 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:32.948338 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697cc88-2995-47ec-a058-49f35418e931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:32.948587 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697cc88-2995-47ec-a058-49f35418e931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:32.959080 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697cc88-2995-47ec-a058-49f35418e931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:32.960556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697cc88-2995-47ec-a058-49f35418e931 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:32.970766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1402/2804] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:43:33.414511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb64ad-182f-48d2-b5e4-a1b69c7c622b None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:33.416057 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f7cb64ad-182f-48d2-b5e4-a1b69c7c622b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:33.432713 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb64ad-182f-48d2-b5e4-a1b69c7c622b None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:33.438618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:33.439643 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:33.441262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:33.441262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] PATH_INFO: `/v3/projects/ff9961b5ad944a09bf66715de7c32e95` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:33.442942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff9961b5ad944a09bf66715de7c32e95)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:33.445290 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:33.569743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff9961b5ad944a09bf66715de7c32e95', 'request_id': 'req-80a074a8-0768-4478-a680-e13d01f49692'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:33.575232 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff9961b5ad944a09bf66715de7c32e95', 'request_id': 'req-80a074a8-0768-4478-a680-e13d01f49692'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:33.648033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff9961b5ad944a09bf66715de7c32e95', 'request_id': 'req-80a074a8-0768-4478-a680-e13d01f49692'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:33.660124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] The token cache is being invalidate because project ff9961b5ad944a09bf66715de7c32e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:43:33.660435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:33.661061 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-80a074a8-0768-4478-a680-e13d01f49692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:43:33.662632 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1403/2805] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:43:33 2022] DELETE /identity/v3/projects/ff9961b5ad944a09bf66715de7c32e95 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:43:33.668734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7360-87b4-4619-b34f-04998f7be81c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:33.669388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7360-87b4-4619-b34f-04998f7be81c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:33.669388 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a7360-87b4-4619-b34f-04998f7be81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:33.691171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-6e8a7360-87b4-4619-b34f-04998f7be81c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:43:33.693056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-6e8a7360-87b4-4619-b34f-04998f7be81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:33.755171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1403/2806] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:43:33.763308 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675da52f-d11e-471c-9ff3-a5ce10e2f619 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:33.764786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675da52f-d11e-471c-9ff3-a5ce10e2f619 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:33.764786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675da52f-d11e-471c-9ff3-a5ce10e2f619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:33.791873 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-675da52f-d11e-471c-9ff3-a5ce10e2f619 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:43:33.793422 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-675da52f-d11e-471c-9ff3-a5ce10e2f619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:33.801669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1404/2807] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:43:33.806345 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a7360-87b4-4619-b34f-04998f7be81c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:33.818910 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a7360-87b4-4619-b34f-04998f7be81c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:33.824279 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc8844-bf35-48a2-8335-5ad901f69042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:33.829839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc8844-bf35-48a2-8335-5ad901f69042 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:33.829839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc8844-bf35-48a2-8335-5ad901f69042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:33.829839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc8844-bf35-48a2-8335-5ad901f69042 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:33.829839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc8844-bf35-48a2-8335-5ad901f69042 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:43:33.829839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc8844-bf35-48a2-8335-5ad901f69042 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:33.829839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edc8844-bf35-48a2-8335-5ad901f69042 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:33.842838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1404/2808] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:43:33.848783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675da52f-d11e-471c-9ff3-a5ce10e2f619 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:33.859054 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675da52f-d11e-471c-9ff3-a5ce10e2f619 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:33.865438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b279c-a6ec-48da-bac1-fd94502abf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:33.867412 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b279c-a6ec-48da-bac1-fd94502abf32 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:33.867928 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b279c-a6ec-48da-bac1-fd94502abf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:33.868348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b279c-a6ec-48da-bac1-fd94502abf32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:33.869752 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b279c-a6ec-48da-bac1-fd94502abf32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:33.872183 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b279c-a6ec-48da-bac1-fd94502abf32 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:33.895481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:43:34.021655 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1405/2809] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:43:33 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:43:34.026270 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc8844-bf35-48a2-8335-5ad901f69042 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:34.037247 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc8844-bf35-48a2-8335-5ad901f69042 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:34.050541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:43:34.110660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f884b0d-f9be-4c6c-9c9f-434b3abbd786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:34.111193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884b0d-f9be-4c6c-9c9f-434b3abbd786 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:34.111478 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884b0d-f9be-4c6c-9c9f-434b3abbd786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:34.111792 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f884b0d-f9be-4c6c-9c9f-434b3abbd786 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:34.112447 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f884b0d-f9be-4c6c-9c9f-434b3abbd786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:34.115082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f884b0d-f9be-4c6c-9c9f-434b3abbd786 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:34.241138 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1405/2810] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:43:34 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:43:34.245952 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b279c-a6ec-48da-bac1-fd94502abf32 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:34.256767 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b279c-a6ec-48da-bac1-fd94502abf32 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:34.261497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a64fa-86d5-4495-b0aa-12957e5e61c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:34.266487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a64fa-86d5-4495-b0aa-12957e5e61c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:34.266487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a64fa-86d5-4495-b0aa-12957e5e61c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:34.266487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a64fa-86d5-4495-b0aa-12957e5e61c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:34.266487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a64fa-86d5-4495-b0aa-12957e5e61c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:34.266487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a64fa-86d5-4495-b0aa-12957e5e61c3 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:34.276550 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1406/2811] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:43:34.280951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f884b0d-f9be-4c6c-9c9f-434b3abbd786 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:34.291944 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f884b0d-f9be-4c6c-9c9f-434b3abbd786 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:34.296668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01071b-20c7-41d4-9281-e6bdec7be46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:34.297272 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01071b-20c7-41d4-9281-e6bdec7be46c admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:34.297554 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01071b-20c7-41d4-9281-e6bdec7be46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:34.297828 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01071b-20c7-41d4-9281-e6bdec7be46c admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:34.298484 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01071b-20c7-41d4-9281-e6bdec7be46c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:34.300130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01071b-20c7-41d4-9281-e6bdec7be46c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:34.311993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1406/2812] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:43:34.316744 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a64fa-86d5-4495-b0aa-12957e5e61c3 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:34.336872 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a64fa-86d5-4495-b0aa-12957e5e61c3 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:34.342743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fa76b-9b79-48f5-ad0e-3e7f1ac00eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:34.343485 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fa76b-9b79-48f5-ad0e-3e7f1ac00eea admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:34.343794 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fa76b-9b79-48f5-ad0e-3e7f1ac00eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:34.344074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fa76b-9b79-48f5-ad0e-3e7f1ac00eea admin admin] PATH_INFO: `/v3/projects/b7d43ef0ca36447b8a5fa7da4e1b2557/users/cdc27b1c5ccc401b8feecbb2deb58b99/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:34.360354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fa76b-9b79-48f5-ad0e-3e7f1ac00eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d43ef0ca36447b8a5fa7da4e1b2557, user_id=cdc27b1c5ccc401b8feecbb2deb58b99, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:34.363230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fa76b-9b79-48f5-ad0e-3e7f1ac00eea admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:34.456942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1407/2813] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:43:34 2022] PUT /identity/v3/projects/b7d43ef0ca36447b8a5fa7da4e1b2557/users/cdc27b1c5ccc401b8feecbb2deb58b99/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:43:34.461630 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ae122-927e-4521-9038-2694edcfa177 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:34.461935 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ae122-927e-4521-9038-2694edcfa177 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:34.462177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ae122-927e-4521-9038-2694edcfa177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:34.494642 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-c60ae122-927e-4521-9038-2694edcfa177 None None] MFA Rules not processed for user `cdc27b1c5ccc401b8feecbb2deb58b99`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:43:34.495936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-c60ae122-927e-4521-9038-2694edcfa177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:34.578497 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1407/2814] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:43:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:43:34.584152 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fa76b-9b79-48f5-ad0e-3e7f1ac00eea admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:34.658050 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fa76b-9b79-48f5-ad0e-3e7f1ac00eea admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:34.663741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e91cc-9c75-4e2d-8f73-4abc39bb212e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:34.664365 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e91cc-9c75-4e2d-8f73-4abc39bb212e admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:34.664645 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e91cc-9c75-4e2d-8f73-4abc39bb212e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:34.664921 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e91cc-9c75-4e2d-8f73-4abc39bb212e admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:34.665604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e91cc-9c75-4e2d-8f73-4abc39bb212e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:34.667547 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e91cc-9c75-4e2d-8f73-4abc39bb212e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:34.767745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1408/2815] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:43:34 2022] POST /identity/v3/projects => generated 375 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:43:34.772228 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ae122-927e-4521-9038-2694edcfa177 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:34.783293 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ae122-927e-4521-9038-2694edcfa177 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:34.845342 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dadae5-8d91-4be4-a8df-bda56846e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:34.846404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dadae5-8d91-4be4-a8df-bda56846e1a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:34.846404 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dadae5-8d91-4be4-a8df-bda56846e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:34.846633 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dadae5-8d91-4be4-a8df-bda56846e1a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:34.847163 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dadae5-8d91-4be4-a8df-bda56846e1a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:34.849725 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dadae5-8d91-4be4-a8df-bda56846e1a7 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:35.035209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1408/2816] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:43:34 2022] POST /identity/v3/users => generated 294 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:43:35.040353 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e91cc-9c75-4e2d-8f73-4abc39bb212e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:35.054421 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e91cc-9c75-4e2d-8f73-4abc39bb212e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:35.060663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088bb59-3fd1-4281-a156-7b59ad9ccaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:35.061285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088bb59-3fd1-4281-a156-7b59ad9ccaf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:35.061569 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088bb59-3fd1-4281-a156-7b59ad9ccaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:35.061845 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088bb59-3fd1-4281-a156-7b59ad9ccaf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:35.062563 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088bb59-3fd1-4281-a156-7b59ad9ccaf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:35.064260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088bb59-3fd1-4281-a156-7b59ad9ccaf6 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:35.076310 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1409/2817] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:43:35.083726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dadae5-8d91-4be4-a8df-bda56846e1a7 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:35.094420 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dadae5-8d91-4be4-a8df-bda56846e1a7 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:35.099989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842244de-407c-42dd-a917-0f906208fb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:35.100922 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842244de-407c-42dd-a917-0f906208fb19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:35.101203 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842244de-407c-42dd-a917-0f906208fb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:35.101481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842244de-407c-42dd-a917-0f906208fb19 admin admin] PATH_INFO: `/v3/projects/50e04fcd78da4295a5fe4b2c9ef90744/users/3a6dfb3334284cf39df31c721c2b7980/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:35.105128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:43:35.118258 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842244de-407c-42dd-a917-0f906208fb19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e04fcd78da4295a5fe4b2c9ef90744, user_id=3a6dfb3334284cf39df31c721c2b7980, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:35.121169 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842244de-407c-42dd-a917-0f906208fb19 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:35.257711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1409/2818] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:43:35 2022] PUT /identity/v3/projects/50e04fcd78da4295a5fe4b2c9ef90744/users/3a6dfb3334284cf39df31c721c2b7980/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:43:35.262092 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088bb59-3fd1-4281-a156-7b59ad9ccaf6 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:35.278640 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088bb59-3fd1-4281-a156-7b59ad9ccaf6 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:35.358672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff539d-2e37-4579-953c-f13e1acf76de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:35.359701 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff539d-2e37-4579-953c-f13e1acf76de admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:35.360203 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff539d-2e37-4579-953c-f13e1acf76de admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:35.360700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff539d-2e37-4579-953c-f13e1acf76de admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:35.361924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff539d-2e37-4579-953c-f13e1acf76de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:35.365190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff539d-2e37-4579-953c-f13e1acf76de admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:35.385898 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1410/2819] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:43:35.390492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842244de-407c-42dd-a917-0f906208fb19 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:35.402424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842244de-407c-42dd-a917-0f906208fb19 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:35.407432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf13735-9119-473d-971c-8e6e08ad988f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:35.408152 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf13735-9119-473d-971c-8e6e08ad988f admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:35.408431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf13735-9119-473d-971c-8e6e08ad988f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:35.408735 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf13735-9119-473d-971c-8e6e08ad988f admin admin] PATH_INFO: `/v3/projects/50e04fcd78da4295a5fe4b2c9ef90744/users/3a6dfb3334284cf39df31c721c2b7980/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:35.410843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf13735-9119-473d-971c-8e6e08ad988f admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e04fcd78da4295a5fe4b2c9ef90744, user_id=3a6dfb3334284cf39df31c721c2b7980, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:35.413843 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf13735-9119-473d-971c-8e6e08ad988f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:35.425373 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1410/2820] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:43:35 2022] PUT /identity/v3/projects/50e04fcd78da4295a5fe4b2c9ef90744/users/3a6dfb3334284cf39df31c721c2b7980/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:43:35.436110 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c5f8-ebaa-493f-a097-cde15eacca6d None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:35.436441 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c5f8-ebaa-493f-a097-cde15eacca6d None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:35.436756 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999c5f8-ebaa-493f-a097-cde15eacca6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:35.472102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-9999c5f8-ebaa-493f-a097-cde15eacca6d None None] MFA Rules not processed for user `3a6dfb3334284cf39df31c721c2b7980`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:43:35.473401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9999c5f8-ebaa-493f-a097-cde15eacca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:35.491083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:43:35.545312 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1411/2821] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:43:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:43:35.556884 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf13735-9119-473d-971c-8e6e08ad988f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:35.558627 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-8bf13735-9119-473d-971c-8e6e08ad988f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:35.621361 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf13735-9119-473d-971c-8e6e08ad988f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:35.625242 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ba85a-7a77-4e84-a7ef-af6a10a97c58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:35.626872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ba85a-7a77-4e84-a7ef-af6a10a97c58 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:35.626872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ba85a-7a77-4e84-a7ef-af6a10a97c58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:35.626872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ba85a-7a77-4e84-a7ef-af6a10a97c58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:35.690124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ba85a-7a77-4e84-a7ef-af6a10a97c58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:35.692074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ba85a-7a77-4e84-a7ef-af6a10a97c58 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:35.702024 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1411/2822] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:43:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:43:42.493367 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999c5f8-ebaa-493f-a097-cde15eacca6d None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:43:42.494882 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9999c5f8-ebaa-493f-a097-cde15eacca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:42.507700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999c5f8-ebaa-493f-a097-cde15eacca6d None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:43:42.511307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32749663-399d-4a02-86de-53e53d454994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:43:42.511977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32749663-399d-4a02-86de-53e53d454994 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:42.512260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32749663-399d-4a02-86de-53e53d454994 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:42.512536 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32749663-399d-4a02-86de-53e53d454994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:42.521669 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32749663-399d-4a02-86de-53e53d454994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:43:42.523102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32749663-399d-4a02-86de-53e53d454994 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:43:42.532607 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1412/2823] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:43:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:43:45.697712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=59,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:43:52.528424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=27 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:43:57.109733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ccc4a-95bd-4d69-9d72-86ec4b9aee8f None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:43:57.109733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ccc4a-95bd-4d69-9d72-86ec4b9aee8f None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:43:57.109733 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ccc4a-95bd-4d69-9d72-86ec4b9aee8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:43:57.124438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-df5ccc4a-95bd-4d69-9d72-86ec4b9aee8f None None] MFA Rules not processed for user `ad7c407cfe924b7a8652f8387c7f77b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:43:57.125949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-df5ccc4a-95bd-4d69-9d72-86ec4b9aee8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:43:57.183969 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1412/2824] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:43:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:01.611678 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32749663-399d-4a02-86de-53e53d454994 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:01.611678 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-32749663-399d-4a02-86de-53e53d454994 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:01.675406 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32749663-399d-4a02-86de-53e53d454994 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:01.681782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002d8e4-e855-4ddd-90aa-dc9a9df433be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:01.683075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002d8e4-e855-4ddd-90aa-dc9a9df433be service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:01.683075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002d8e4-e855-4ddd-90aa-dc9a9df433be service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:01.683075 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002d8e4-e855-4ddd-90aa-dc9a9df433be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:01.684998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-f002d8e4-e855-4ddd-90aa-dc9a9df433be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:01.697165 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002d8e4-e855-4ddd-90aa-dc9a9df433be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:01.698313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002d8e4-e855-4ddd-90aa-dc9a9df433be service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:01.707739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1413/2825] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:44:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:44:07.176464 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=25 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:11.703408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=34 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:12.403696 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ccc4a-95bd-4d69-9d72-86ec4b9aee8f None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:12.468159 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ccc4a-95bd-4d69-9d72-86ec4b9aee8f None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:12.472823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:12.473463 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:12.473747 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:12.474018 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] PATH_INFO: `/v3/users/cdc27b1c5ccc401b8feecbb2deb58b99` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:12.475160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdc27b1c5ccc401b8feecbb2deb58b99)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:12.477375 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:12.704306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdc27b1c5ccc401b8feecbb2deb58b99'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:12.704972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdc27b1c5ccc401b8feecbb2deb58b99'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:12.705602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdc27b1c5ccc401b8feecbb2deb58b99'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:12.712781 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdc27b1c5ccc401b8feecbb2deb58b99'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:12.726079 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdc27b1c5ccc401b8feecbb2deb58b99'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:12.811284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1413/2826] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:44:12 2022] DELETE /identity/v3/users/cdc27b1c5ccc401b8feecbb2deb58b99 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:44:12.816097 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1121c6-e7ee-46e8-be68-e85eeaee8306 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:12.816597 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1121c6-e7ee-46e8-be68-e85eeaee8306 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:12.816704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1121c6-e7ee-46e8-be68-e85eeaee8306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:12.836754 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-9b1121c6-e7ee-46e8-be68-e85eeaee8306 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:12.838029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9b1121c6-e7ee-46e8-be68-e85eeaee8306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:12.904844 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1414/2827] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:12.914885 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:12.916853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:12.976536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc74198-9420-4ed9-a63f-4c8bfb13296e admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:12.980670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee1072-e663-48b6-99cf-635fa9754466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:12.981289 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee1072-e663-48b6-99cf-635fa9754466 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:12.981573 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee1072-e663-48b6-99cf-635fa9754466 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:12.981851 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee1072-e663-48b6-99cf-635fa9754466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:12.991809 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee1072-e663-48b6-99cf-635fa9754466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:12.993282 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee1072-e663-48b6-99cf-635fa9754466 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:13.004074 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1414/2828] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:44:13.480044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1121c6-e7ee-46e8-be68-e85eeaee8306 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:13.481502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9b1121c6-e7ee-46e8-be68-e85eeaee8306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:13.494305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1121c6-e7ee-46e8-be68-e85eeaee8306 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:13.499221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:13.499971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:13.500515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:13.500682 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] PATH_INFO: `/v3/projects/b7d43ef0ca36447b8a5fa7da4e1b2557` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:13.501893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d43ef0ca36447b8a5fa7da4e1b2557)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:13.503909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:13.624620 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d43ef0ca36447b8a5fa7da4e1b2557', 'request_id': 'req-149b2b7a-ce10-47d0-9573-418011aaf801'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:13.630076 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d43ef0ca36447b8a5fa7da4e1b2557', 'request_id': 'req-149b2b7a-ce10-47d0-9573-418011aaf801'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:13.732264 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d43ef0ca36447b8a5fa7da4e1b2557', 'request_id': 'req-149b2b7a-ce10-47d0-9573-418011aaf801'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:13.745213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] The token cache is being invalidate because project b7d43ef0ca36447b8a5fa7da4e1b2557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:44:13.745491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:13.746149 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:13.747670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1415/2829] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:44:13 2022] DELETE /identity/v3/projects/b7d43ef0ca36447b8a5fa7da4e1b2557 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:13.751945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee1072-e663-48b6-99cf-635fa9754466 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:13.762226 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee1072-e663-48b6-99cf-635fa9754466 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:13.781017 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:13.828111 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:13.829178 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:13.829660 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:13.830001 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] PATH_INFO: `/v3/users/3a6dfb3334284cf39df31c721c2b7980` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:13.831324 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a6dfb3334284cf39df31c721c2b7980)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:13.833471 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:14.128004 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a6dfb3334284cf39df31c721c2b7980'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:14.128775 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a6dfb3334284cf39df31c721c2b7980'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:14.129384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a6dfb3334284cf39df31c721c2b7980'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:14.137178 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a6dfb3334284cf39df31c721c2b7980'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:14.153457 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-d5b80a5e-752f-475a-9faa-38df63b496cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a6dfb3334284cf39df31c721c2b7980'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:14.251335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1415/2830] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:44:13 2022] DELETE /identity/v3/users/3a6dfb3334284cf39df31c721c2b7980 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:14.739368 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:14.741602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:14.755182 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b2b7a-ce10-47d0-9573-418011aaf801 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:14.821108 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:14.821747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:14.822181 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:14.822401 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] PATH_INFO: `/v3/projects/50e04fcd78da4295a5fe4b2c9ef90744` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:14.823483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e04fcd78da4295a5fe4b2c9ef90744)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:14.825455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:14.955455 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:14.973028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e04fcd78da4295a5fe4b2c9ef90744', 'request_id': 'req-eda1d18e-5ef5-4269-a946-5f18314afc3f'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:14.978737 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e04fcd78da4295a5fe4b2c9ef90744', 'request_id': 'req-eda1d18e-5ef5-4269-a946-5f18314afc3f'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:15.030447 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e04fcd78da4295a5fe4b2c9ef90744', 'request_id': 'req-eda1d18e-5ef5-4269-a946-5f18314afc3f'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:15.042704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] The token cache is being invalidate because project 50e04fcd78da4295a5fe4b2c9ef90744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:44:15.042973 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:15.043726 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-eda1d18e-5ef5-4269-a946-5f18314afc3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:15.045224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1416/2831] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:44:14 2022] DELETE /identity/v3/projects/50e04fcd78da4295a5fe4b2c9ef90744 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:15.050360 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc6a86-8738-4b01-8709-82e90a868320 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:15.050880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc6a86-8738-4b01-8709-82e90a868320 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:15.050880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc6a86-8738-4b01-8709-82e90a868320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:15.071913 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-69dc6a86-8738-4b01-8709-82e90a868320 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:15.073140 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-69dc6a86-8738-4b01-8709-82e90a868320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:15.134793 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1416/2832] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:15.199015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1e4dd-3b1c-46f5-8bff-a498f68a994c None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:15.199299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1e4dd-3b1c-46f5-8bff-a498f68a994c None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:15.199725 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b1e4dd-3b1c-46f5-8bff-a498f68a994c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:15.220268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-b0b1e4dd-3b1c-46f5-8bff-a498f68a994c None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:15.221484 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b0b1e4dd-3b1c-46f5-8bff-a498f68a994c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:15.229959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1417/2833] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:15.234957 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc6a86-8738-4b01-8709-82e90a868320 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:15.244600 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc6a86-8738-4b01-8709-82e90a868320 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:15.249651 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65352c3-0638-4dbf-8b54-52ed78a33e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:15.250256 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65352c3-0638-4dbf-8b54-52ed78a33e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:15.250545 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65352c3-0638-4dbf-8b54-52ed78a33e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:15.250808 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65352c3-0638-4dbf-8b54-52ed78a33e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:15.251458 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65352c3-0638-4dbf-8b54-52ed78a33e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:44:15.251789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65352c3-0638-4dbf-8b54-52ed78a33e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:15.253387 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65352c3-0638-4dbf-8b54-52ed78a33e03 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:15.265975 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1417/2834] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:44:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:44:15.270575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1e4dd-3b1c-46f5-8bff-a498f68a994c None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:15.281453 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b1e4dd-3b1c-46f5-8bff-a498f68a994c None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:15.286603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6efdd-0baf-472d-ae1a-1829c4a73661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:15.287229 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6efdd-0baf-472d-ae1a-1829c4a73661 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:15.287510 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6efdd-0baf-472d-ae1a-1829c4a73661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:15.287823 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6efdd-0baf-472d-ae1a-1829c4a73661 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:15.288460 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6efdd-0baf-472d-ae1a-1829c4a73661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:15.290435 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6efdd-0baf-472d-ae1a-1829c4a73661 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:15.386376 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1418/2835] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:44:15 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:44:15.391346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65352c3-0638-4dbf-8b54-52ed78a33e03 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:15.403082 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65352c3-0638-4dbf-8b54-52ed78a33e03 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:15.430951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:15.464216 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1896d9a-804b-4797-85a4-6d22e5d13aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:15.464838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1896d9a-804b-4797-85a4-6d22e5d13aba admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:15.465124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1896d9a-804b-4797-85a4-6d22e5d13aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:15.465403 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1896d9a-804b-4797-85a4-6d22e5d13aba admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:15.466049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1896d9a-804b-4797-85a4-6d22e5d13aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:15.468034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1896d9a-804b-4797-85a4-6d22e5d13aba admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:15.567710 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1418/2836] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:44:15 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:44:15.571724 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6efdd-0baf-472d-ae1a-1829c4a73661 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:15.582636 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6efdd-0baf-472d-ae1a-1829c4a73661 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:15.587285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436daea4-5fc7-40df-b732-1603a8e084b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:15.587913 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436daea4-5fc7-40df-b732-1603a8e084b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:15.588197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436daea4-5fc7-40df-b732-1603a8e084b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:15.588473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436daea4-5fc7-40df-b732-1603a8e084b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:15.589123 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436daea4-5fc7-40df-b732-1603a8e084b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:15.590723 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436daea4-5fc7-40df-b732-1603a8e084b3 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:15.601038 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1419/2837] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:44:15.605413 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1896d9a-804b-4797-85a4-6d22e5d13aba admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:15.615358 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1896d9a-804b-4797-85a4-6d22e5d13aba admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:15.620015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d01f7-b457-4a19-85bf-a172915b0845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:15.620608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d01f7-b457-4a19-85bf-a172915b0845 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:15.620889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d01f7-b457-4a19-85bf-a172915b0845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:15.621167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d01f7-b457-4a19-85bf-a172915b0845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:15.622317 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d01f7-b457-4a19-85bf-a172915b0845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:15.624016 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d01f7-b457-4a19-85bf-a172915b0845 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:15.634783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1419/2838] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:44:15.639822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436daea4-5fc7-40df-b732-1603a8e084b3 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:15.649932 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436daea4-5fc7-40df-b732-1603a8e084b3 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:15.654940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcb64b-c1bb-43e5-acd3-1f0d2e2f9d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:15.655627 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcb64b-c1bb-43e5-acd3-1f0d2e2f9d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:15.655914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcb64b-c1bb-43e5-acd3-1f0d2e2f9d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:15.656190 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcb64b-c1bb-43e5-acd3-1f0d2e2f9d16 admin admin] PATH_INFO: `/v3/projects/8bf66c9f64f04a4593bbcead288716cf/users/4fedf0eb4e614a8991889d56f9f546fc/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:15.671438 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcb64b-c1bb-43e5-acd3-1f0d2e2f9d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf66c9f64f04a4593bbcead288716cf, user_id=4fedf0eb4e614a8991889d56f9f546fc, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:15.674155 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcb64b-c1bb-43e5-acd3-1f0d2e2f9d16 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:15.748179 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1420/2839] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:44:15 2022] PUT /identity/v3/projects/8bf66c9f64f04a4593bbcead288716cf/users/4fedf0eb4e614a8991889d56f9f546fc/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:15.758432 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2245cca-0769-4e80-ae5c-e47bf38de60e None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:15.758751 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2245cca-0769-4e80-ae5c-e47bf38de60e None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:15.759032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2245cca-0769-4e80-ae5c-e47bf38de60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:15.788683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b2245cca-0769-4e80-ae5c-e47bf38de60e None None] MFA Rules not processed for user `4fedf0eb4e614a8991889d56f9f546fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:15.789925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b2245cca-0769-4e80-ae5c-e47bf38de60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:15.904508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1420/2840] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:15.909098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcb64b-c1bb-43e5-acd3-1f0d2e2f9d16 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:15.976052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcb64b-c1bb-43e5-acd3-1f0d2e2f9d16 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:15.981049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a5a33-05cd-4d3f-8de8-3fcf3f3b6d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:15.981672 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5a33-05cd-4d3f-8de8-3fcf3f3b6d35 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:15.981959 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5a33-05cd-4d3f-8de8-3fcf3f3b6d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:15.982317 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a5a33-05cd-4d3f-8de8-3fcf3f3b6d35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:15.983490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a5a33-05cd-4d3f-8de8-3fcf3f3b6d35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:15.985430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a5a33-05cd-4d3f-8de8-3fcf3f3b6d35 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:16.069683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1421/2841] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:44:15 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:44:16.074214 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2245cca-0769-4e80-ae5c-e47bf38de60e None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:16.086179 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2245cca-0769-4e80-ae5c-e47bf38de60e None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:16.150002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f8e85-fc0a-4112-a9d6-9b9f2be380a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:16.150607 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f8e85-fc0a-4112-a9d6-9b9f2be380a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:16.157916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f8e85-fc0a-4112-a9d6-9b9f2be380a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:16.157916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f8e85-fc0a-4112-a9d6-9b9f2be380a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:16.157916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f8e85-fc0a-4112-a9d6-9b9f2be380a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:16.157916 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f8e85-fc0a-4112-a9d6-9b9f2be380a8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:16.258906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1421/2842] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:44:16 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:44:16.262997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a5a33-05cd-4d3f-8de8-3fcf3f3b6d35 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:16.275511 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a5a33-05cd-4d3f-8de8-3fcf3f3b6d35 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:16.280318 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337b53-0629-4c11-9a08-c41853a8452a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:16.280929 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337b53-0629-4c11-9a08-c41853a8452a admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:16.281262 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337b53-0629-4c11-9a08-c41853a8452a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:16.281634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337b53-0629-4c11-9a08-c41853a8452a admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:16.282285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d337b53-0629-4c11-9a08-c41853a8452a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:16.283944 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d337b53-0629-4c11-9a08-c41853a8452a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:16.295695 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1422/2843] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:44:16.299987 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f8e85-fc0a-4112-a9d6-9b9f2be380a8 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:16.310127 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f8e85-fc0a-4112-a9d6-9b9f2be380a8 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:16.315859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f7067-be66-4946-869d-89acaecb8a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:16.316544 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f7067-be66-4946-869d-89acaecb8a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:16.316830 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f7067-be66-4946-869d-89acaecb8a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:16.317113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f7067-be66-4946-869d-89acaecb8a72 admin admin] PATH_INFO: `/v3/projects/901bcfb552b44bce9dd2783914f8e803/users/7cb7b682595441ee94a155d219b413d6/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:16.334553 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f7067-be66-4946-869d-89acaecb8a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901bcfb552b44bce9dd2783914f8e803, user_id=7cb7b682595441ee94a155d219b413d6, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:16.338551 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f7067-be66-4946-869d-89acaecb8a72 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:16.462142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1422/2844] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:44:16 2022] PUT /identity/v3/projects/901bcfb552b44bce9dd2783914f8e803/users/7cb7b682595441ee94a155d219b413d6/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:16.466147 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337b53-0629-4c11-9a08-c41853a8452a admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:16.476739 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337b53-0629-4c11-9a08-c41853a8452a admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:16.497571 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:16.534789 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d513aa-25a2-4d97-8418-b5af830f4b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:16.535821 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d513aa-25a2-4d97-8418-b5af830f4b07 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:16.536112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d513aa-25a2-4d97-8418-b5af830f4b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:16.536411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d513aa-25a2-4d97-8418-b5af830f4b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:16.537096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d513aa-25a2-4d97-8418-b5af830f4b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:16.538697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d513aa-25a2-4d97-8418-b5af830f4b07 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:16.549521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1423/2845] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:44:16.553789 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f7067-be66-4946-869d-89acaecb8a72 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:16.559355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:16.565566 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f7067-be66-4946-869d-89acaecb8a72 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:16.570445 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae8c5-dcaf-4e91-ac62-995ff5153faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:16.571111 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae8c5-dcaf-4e91-ac62-995ff5153faa admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:16.571424 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae8c5-dcaf-4e91-ac62-995ff5153faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:16.571728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae8c5-dcaf-4e91-ac62-995ff5153faa admin admin] PATH_INFO: `/v3/projects/901bcfb552b44bce9dd2783914f8e803/users/7cb7b682595441ee94a155d219b413d6/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:16.573683 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae8c5-dcaf-4e91-ac62-995ff5153faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=901bcfb552b44bce9dd2783914f8e803, user_id=7cb7b682595441ee94a155d219b413d6, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:16.576490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae8c5-dcaf-4e91-ac62-995ff5153faa admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:16.587292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1423/2846] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:44:16 2022] PUT /identity/v3/projects/901bcfb552b44bce9dd2783914f8e803/users/7cb7b682595441ee94a155d219b413d6/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:16.601650 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbb2df-4b1f-4e09-ad4a-f45cc3fcc625 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:16.601881 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbb2df-4b1f-4e09-ad4a-f45cc3fcc625 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:16.602191 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbb2df-4b1f-4e09-ad4a-f45cc3fcc625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:16.633109 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-6adbb2df-4b1f-4e09-ad4a-f45cc3fcc625 None None] MFA Rules not processed for user `7cb7b682595441ee94a155d219b413d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:16.634414 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6adbb2df-4b1f-4e09-ad4a-f45cc3fcc625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:16.703525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1424/2847] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:16.716347 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae8c5-dcaf-4e91-ac62-995ff5153faa admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:16.718688 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f5bae8c5-dcaf-4e91-ac62-995ff5153faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:16.781770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae8c5-dcaf-4e91-ac62-995ff5153faa admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:16.785574 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1a936-ca69-4bbc-abd6-8530766ddf59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:16.786190 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1a936-ca69-4bbc-abd6-8530766ddf59 service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:16.786467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1a936-ca69-4bbc-abd6-8530766ddf59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:16.786738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1a936-ca69-4bbc-abd6-8530766ddf59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:16.797198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1a936-ca69-4bbc-abd6-8530766ddf59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:16.799434 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1a936-ca69-4bbc-abd6-8530766ddf59 service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:16.812541 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1424/2848] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:44:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:44:16.880489 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbb2df-4b1f-4e09-ad4a-f45cc3fcc625 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:16.977391 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbb2df-4b1f-4e09-ad4a-f45cc3fcc625 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:16.982028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:16.983314 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:16.983617 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:16.983890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] PATH_INFO: `/v3/users/4fedf0eb4e614a8991889d56f9f546fc` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:16.985044 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fedf0eb4e614a8991889d56f9f546fc)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:16.986972 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:17.236828 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fedf0eb4e614a8991889d56f9f546fc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:17.238590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:17.315729 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fedf0eb4e614a8991889d56f9f546fc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:17.316371 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fedf0eb4e614a8991889d56f9f546fc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:17.329886 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fedf0eb4e614a8991889d56f9f546fc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:17.330584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fedf0eb4e614a8991889d56f9f546fc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:17.343942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1425/2849] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:44:16 2022] DELETE /identity/v3/users/4fedf0eb4e614a8991889d56f9f546fc => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:17.348690 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e28943-0287-44fe-8032-53dfabe522c7 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:17.349184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e28943-0287-44fe-8032-53dfabe522c7 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:17.349302 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e28943-0287-44fe-8032-53dfabe522c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:17.369459 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-45e28943-0287-44fe-8032-53dfabe522c7 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:17.370711 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-45e28943-0287-44fe-8032-53dfabe522c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:17.434532 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1425/2850] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:17.445405 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:17.450028 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:17.512705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c85bf-958d-4f4d-95ba-200b457ffe62 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:17.516602 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f15c26-212f-477c-a74a-50de3e85e3dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:17.517261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f15c26-212f-477c-a74a-50de3e85e3dd service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:17.517497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f15c26-212f-477c-a74a-50de3e85e3dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:17.517771 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f15c26-212f-477c-a74a-50de3e85e3dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:17.528719 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f15c26-212f-477c-a74a-50de3e85e3dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:17.530531 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f15c26-212f-477c-a74a-50de3e85e3dd service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:17.540557 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1426/2851] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:44:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:44:17.972596 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e28943-0287-44fe-8032-53dfabe522c7 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:17.974800 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-45e28943-0287-44fe-8032-53dfabe522c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:17.990509 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e28943-0287-44fe-8032-53dfabe522c7 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:17.996331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:17.997095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:17.997365 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:17.997657 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] PATH_INFO: `/v3/projects/8bf66c9f64f04a4593bbcead288716cf` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:17.999049 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf66c9f64f04a4593bbcead288716cf)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:18.001040 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:18.105583 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf66c9f64f04a4593bbcead288716cf', 'request_id': 'req-5592da89-c757-48d1-85ee-819f1c8e481e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:18.111880 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf66c9f64f04a4593bbcead288716cf', 'request_id': 'req-5592da89-c757-48d1-85ee-819f1c8e481e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:18.198674 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf66c9f64f04a4593bbcead288716cf', 'request_id': 'req-5592da89-c757-48d1-85ee-819f1c8e481e'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:18.213440 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] The token cache is being invalidate because project 8bf66c9f64f04a4593bbcead288716cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:44:18.213736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:18.214472 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:18.215972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1426/2852] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:44:17 2022] DELETE /identity/v3/projects/8bf66c9f64f04a4593bbcead288716cf => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:18.220647 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f15c26-212f-477c-a74a-50de3e85e3dd service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:18.231197 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f15c26-212f-477c-a74a-50de3e85e3dd service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:18.294546 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:18.295354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:18.295779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:18.296299 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] PATH_INFO: `/v3/users/7cb7b682595441ee94a155d219b413d6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:18.297641 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb7b682595441ee94a155d219b413d6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:18.303224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:18.507834 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb7b682595441ee94a155d219b413d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:18.598566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb7b682595441ee94a155d219b413d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:18.599192 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb7b682595441ee94a155d219b413d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:18.612526 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:18.614194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb7b682595441ee94a155d219b413d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:18.614890 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-587ea5ad-758d-4fa8-bf92-6adf0440a253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb7b682595441ee94a155d219b413d6'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:18.628261 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1427/2853] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:44:18 2022] DELETE /identity/v3/users/7cb7b682595441ee94a155d219b413d6 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:19.106231 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:19.108410 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:19.121895 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592da89-c757-48d1-85ee-819f1c8e481e admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:19.135778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:19.184251 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:19.184891 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:19.185174 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:19.185453 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] PATH_INFO: `/v3/projects/901bcfb552b44bce9dd2783914f8e803` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:19.186719 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901bcfb552b44bce9dd2783914f8e803)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:19.188749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:19.298456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901bcfb552b44bce9dd2783914f8e803', 'request_id': 'req-845328d7-4c1a-479a-a3ba-096bef9483e8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:19.304764 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901bcfb552b44bce9dd2783914f8e803', 'request_id': 'req-845328d7-4c1a-479a-a3ba-096bef9483e8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:19.372852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901bcfb552b44bce9dd2783914f8e803', 'request_id': 'req-845328d7-4c1a-479a-a3ba-096bef9483e8'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:19.385818 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] The token cache is being invalidate because project 901bcfb552b44bce9dd2783914f8e803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:44:19.386112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:19.386785 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-845328d7-4c1a-479a-a3ba-096bef9483e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:19.390475 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1427/2854] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:44:19 2022] DELETE /identity/v3/projects/901bcfb552b44bce9dd2783914f8e803 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:19.395665 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5babde-24bc-40bc-beda-7d5e45991fb0 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:19.396219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5babde-24bc-40bc-beda-7d5e45991fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:19.396219 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5babde-24bc-40bc-beda-7d5e45991fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:19.421364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8a5babde-24bc-40bc-beda-7d5e45991fb0 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:19.422581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8a5babde-24bc-40bc-beda-7d5e45991fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:19.489513 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1428/2855] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:19.497602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd69721-2d55-4a50-84d6-d91428f61603 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:19.497886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd69721-2d55-4a50-84d6-d91428f61603 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:19.498167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd69721-2d55-4a50-84d6-d91428f61603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:19.519267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-cdd69721-2d55-4a50-84d6-d91428f61603 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:19.520584 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-cdd69721-2d55-4a50-84d6-d91428f61603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:19.529186 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1428/2856] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:19.534166 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5babde-24bc-40bc-beda-7d5e45991fb0 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:19.554666 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5babde-24bc-40bc-beda-7d5e45991fb0 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:19.560634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8cb96-a013-4569-bd32-d4ba8ace6781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:19.561268 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8cb96-a013-4569-bd32-d4ba8ace6781 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:19.561549 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8cb96-a013-4569-bd32-d4ba8ace6781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:19.561824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe8cb96-a013-4569-bd32-d4ba8ace6781 admin admin] PATH_INFO: `/v3/domains` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:19.562791 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8cb96-a013-4569-bd32-d4ba8ace6781 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81961) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:44:19.563112 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8cb96-a013-4569-bd32-d4ba8ace6781 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:19.564775 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe8cb96-a013-4569-bd32-d4ba8ace6781 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:19.580154 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1429/2857] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:44:19.584838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd69721-2d55-4a50-84d6-d91428f61603 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:19.595889 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd69721-2d55-4a50-84d6-d91428f61603 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:19.600949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829da4b9-94a8-4599-af7a-82de00c18a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:19.601561 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829da4b9-94a8-4599-af7a-82de00c18a31 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:19.601839 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829da4b9-94a8-4599-af7a-82de00c18a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:19.602117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829da4b9-94a8-4599-af7a-82de00c18a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:19.602834 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829da4b9-94a8-4599-af7a-82de00c18a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:19.604777 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829da4b9-94a8-4599-af7a-82de00c18a31 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:19.695099 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1429/2858] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:44:19 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:44:19.699390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8cb96-a013-4569-bd32-d4ba8ace6781 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:19.710213 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe8cb96-a013-4569-bd32-d4ba8ace6781 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:19.768323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc93bb5-fa9c-4887-b08c-c260db9cb13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:19.768936 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc93bb5-fa9c-4887-b08c-c260db9cb13c admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:19.769216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc93bb5-fa9c-4887-b08c-c260db9cb13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:19.769491 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc93bb5-fa9c-4887-b08c-c260db9cb13c admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:19.770161 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc93bb5-fa9c-4887-b08c-c260db9cb13c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:19.772160 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc93bb5-fa9c-4887-b08c-c260db9cb13c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:19.876977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1430/2859] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:44:19 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:44:19.881421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829da4b9-94a8-4599-af7a-82de00c18a31 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:19.893184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829da4b9-94a8-4599-af7a-82de00c18a31 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:19.898575 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c051cbc-5d7f-4f35-8b05-ba1ddacc120f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:19.899193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c051cbc-5d7f-4f35-8b05-ba1ddacc120f admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:19.899507 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c051cbc-5d7f-4f35-8b05-ba1ddacc120f admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:19.899823 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c051cbc-5d7f-4f35-8b05-ba1ddacc120f admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:19.900486 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c051cbc-5d7f-4f35-8b05-ba1ddacc120f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:19.902139 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c051cbc-5d7f-4f35-8b05-ba1ddacc120f admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:19.913070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1430/2860] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:44:19.917313 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc93bb5-fa9c-4887-b08c-c260db9cb13c admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:19.928062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc93bb5-fa9c-4887-b08c-c260db9cb13c admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:19.933319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593a9d8a-1076-40d5-888d-c2021c34713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:19.937194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9d8a-1076-40d5-888d-c2021c34713c admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:19.937194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9d8a-1076-40d5-888d-c2021c34713c admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:19.937194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593a9d8a-1076-40d5-888d-c2021c34713c admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:19.937194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a9d8a-1076-40d5-888d-c2021c34713c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:19.937584 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593a9d8a-1076-40d5-888d-c2021c34713c admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:19.948382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1431/2861] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:44:19.953276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c051cbc-5d7f-4f35-8b05-ba1ddacc120f admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:19.963736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c051cbc-5d7f-4f35-8b05-ba1ddacc120f admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:19.968709 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de42770-b633-453f-afa4-b76fa7e09417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:19.969389 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de42770-b633-453f-afa4-b76fa7e09417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:19.969670 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de42770-b633-453f-afa4-b76fa7e09417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:19.969952 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de42770-b633-453f-afa4-b76fa7e09417 admin admin] PATH_INFO: `/v3/projects/7f96e3362a23448394155a97547f9b90/users/a5ec98a403dc4c459bb66dc4236f871a/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:19.985536 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de42770-b633-453f-afa4-b76fa7e09417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f96e3362a23448394155a97547f9b90, user_id=a5ec98a403dc4c459bb66dc4236f871a, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:19.988239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de42770-b633-453f-afa4-b76fa7e09417 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:20.078076 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1431/2862] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:44:19 2022] PUT /identity/v3/projects/7f96e3362a23448394155a97547f9b90/users/a5ec98a403dc4c459bb66dc4236f871a/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:20.088784 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ca6d9-c28f-4112-a3ee-881f2447f07a None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:20.089015 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ca6d9-c28f-4112-a3ee-881f2447f07a None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:20.089288 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ca6d9-c28f-4112-a3ee-881f2447f07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:20.118349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-065ca6d9-c28f-4112-a3ee-881f2447f07a None None] MFA Rules not processed for user `a5ec98a403dc4c459bb66dc4236f871a`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:20.119604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-065ca6d9-c28f-4112-a3ee-881f2447f07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:20.164809 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:20.188670 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1432/2863] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:20.193525 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de42770-b633-453f-afa4-b76fa7e09417 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:20.242567 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:20.265262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de42770-b633-453f-afa4-b76fa7e09417 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:20.272576 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b722a-1272-4da0-aed8-40146e84e387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:20.272852 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b722a-1272-4da0-aed8-40146e84e387 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:20.273222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b722a-1272-4da0-aed8-40146e84e387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:20.273706 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b722a-1272-4da0-aed8-40146e84e387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:20.274336 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b722a-1272-4da0-aed8-40146e84e387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:20.276604 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864b722a-1272-4da0-aed8-40146e84e387 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:20.449450 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1432/2864] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:44:20 2022] POST /identity/v3/projects => generated 377 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:44:20.453974 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ca6d9-c28f-4112-a3ee-881f2447f07a None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:20.466163 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ca6d9-c28f-4112-a3ee-881f2447f07a None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:20.525562 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed81c0-1030-4765-aa03-b435d4160df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:20.526072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed81c0-1030-4765-aa03-b435d4160df2 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:20.526346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed81c0-1030-4765-aa03-b435d4160df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:20.526618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed81c0-1030-4765-aa03-b435d4160df2 admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:20.527253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed81c0-1030-4765-aa03-b435d4160df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:20.529711 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed81c0-1030-4765-aa03-b435d4160df2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:20.627445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1433/2865] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:44:20 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:44:20.631520 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b722a-1272-4da0-aed8-40146e84e387 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:20.642396 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b722a-1272-4da0-aed8-40146e84e387 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:20.647045 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316492d-f428-42e8-84a4-9002d3aa5aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:20.647681 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316492d-f428-42e8-84a4-9002d3aa5aff admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:20.647963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316492d-f428-42e8-84a4-9002d3aa5aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:20.648233 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316492d-f428-42e8-84a4-9002d3aa5aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:20.648971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316492d-f428-42e8-84a4-9002d3aa5aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:20.650619 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316492d-f428-42e8-84a4-9002d3aa5aff admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:20.661958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1433/2866] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:44:20.666339 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed81c0-1030-4765-aa03-b435d4160df2 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:20.677997 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed81c0-1030-4765-aa03-b435d4160df2 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:20.682832 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37270ce5-2d71-42f1-933f-635dbbd59a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:20.683439 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37270ce5-2d71-42f1-933f-635dbbd59a45 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:20.683761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37270ce5-2d71-42f1-933f-635dbbd59a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:20.684031 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37270ce5-2d71-42f1-933f-635dbbd59a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:20.684680 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37270ce5-2d71-42f1-933f-635dbbd59a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:20.686322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37270ce5-2d71-42f1-933f-635dbbd59a45 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:20.696747 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1434/2867] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:44:20.701154 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316492d-f428-42e8-84a4-9002d3aa5aff admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:20.711521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316492d-f428-42e8-84a4-9002d3aa5aff admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:20.717349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851f24a-71eb-4ebc-92c9-e48caf4251f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:20.718046 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851f24a-71eb-4ebc-92c9-e48caf4251f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:20.718327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851f24a-71eb-4ebc-92c9-e48caf4251f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:20.718755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851f24a-71eb-4ebc-92c9-e48caf4251f3 admin admin] PATH_INFO: `/v3/projects/4f1439bbfd7d4ac5aefe7c595d8eed6e/users/1bebc347aede4cf6840423e596f165d7/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:20.735123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851f24a-71eb-4ebc-92c9-e48caf4251f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f1439bbfd7d4ac5aefe7c595d8eed6e, user_id=1bebc347aede4cf6840423e596f165d7, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:20.737871 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851f24a-71eb-4ebc-92c9-e48caf4251f3 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:20.839441 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1434/2868] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:44:20 2022] PUT /identity/v3/projects/4f1439bbfd7d4ac5aefe7c595d8eed6e/users/1bebc347aede4cf6840423e596f165d7/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:44:20.852410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c355f-16f4-4fa7-a5b6-990446af4540 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:20.852783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c355f-16f4-4fa7-a5b6-990446af4540 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:20.853158 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c355f-16f4-4fa7-a5b6-990446af4540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:20.883678 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-6c1c355f-16f4-4fa7-a5b6-990446af4540 None None] MFA Rules not processed for user `1bebc347aede4cf6840423e596f165d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:20.885041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6c1c355f-16f4-4fa7-a5b6-990446af4540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:20.958049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1435/2869] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:44:20.962726 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851f24a-71eb-4ebc-92c9-e48caf4251f3 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:21.029010 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851f24a-71eb-4ebc-92c9-e48caf4251f3 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:21.034254 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cc4ea-ef4d-4542-b9c4-8f4e550087b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:21.034872 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc4ea-ef4d-4542-b9c4-8f4e550087b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:21.035151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc4ea-ef4d-4542-b9c4-8f4e550087b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:21.035426 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc4ea-ef4d-4542-b9c4-8f4e550087b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:21.036124 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cc4ea-ef4d-4542-b9c4-8f4e550087b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:21.038048 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cc4ea-ef4d-4542-b9c4-8f4e550087b6 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:21.191897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1435/2870] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:44:20 2022] POST /identity/v3/projects => generated 377 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:44:21.196098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c355f-16f4-4fa7-a5b6-990446af4540 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:21.208396 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c355f-16f4-4fa7-a5b6-990446af4540 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:21.261614 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:21.273199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863dc643-f11e-4af0-9a43-92fe194d8bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:21.273824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dc643-f11e-4af0-9a43-92fe194d8bab admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:21.274107 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dc643-f11e-4af0-9a43-92fe194d8bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:21.274379 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863dc643-f11e-4af0-9a43-92fe194d8bab admin admin] PATH_INFO: `/v3/users` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:21.275019 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863dc643-f11e-4af0-9a43-92fe194d8bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:21.276979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863dc643-f11e-4af0-9a43-92fe194d8bab admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:21.388326 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1436/2871] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:44:21 2022] POST /identity/v3/users => generated 295 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:44:21.393162 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cc4ea-ef4d-4542-b9c4-8f4e550087b6 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:21.404055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cc4ea-ef4d-4542-b9c4-8f4e550087b6 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:21.410015 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6330c-0567-4bdd-a8e0-f4559325afcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:21.410661 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6330c-0567-4bdd-a8e0-f4559325afcd admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:21.410951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6330c-0567-4bdd-a8e0-f4559325afcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:21.411267 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6330c-0567-4bdd-a8e0-f4559325afcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:21.412013 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6330c-0567-4bdd-a8e0-f4559325afcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:21.413763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6330c-0567-4bdd-a8e0-f4559325afcd admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:21.425384 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1436/2872] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:44:21.429616 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863dc643-f11e-4af0-9a43-92fe194d8bab admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:21.440280 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863dc643-f11e-4af0-9a43-92fe194d8bab admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:21.445033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42dd2da-45d7-419e-8746-1fa8ac899025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:21.445682 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dd2da-45d7-419e-8746-1fa8ac899025 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:21.445965 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dd2da-45d7-419e-8746-1fa8ac899025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:21.446234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dd2da-45d7-419e-8746-1fa8ac899025 admin admin] PATH_INFO: `/v3/projects/093322120a7d40d290a1a0e96259a1f1/users/189bf89cfbc44914b2287bf3fe044168/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:21.463234 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42dd2da-45d7-419e-8746-1fa8ac899025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=093322120a7d40d290a1a0e96259a1f1, user_id=189bf89cfbc44914b2287bf3fe044168, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:21.465982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42dd2da-45d7-419e-8746-1fa8ac899025 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:21.549962 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1437/2873] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:44:21 2022] PUT /identity/v3/projects/093322120a7d40d290a1a0e96259a1f1/users/189bf89cfbc44914b2287bf3fe044168/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:21.552736 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6330c-0567-4bdd-a8e0-f4559325afcd admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:21.563816 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6330c-0567-4bdd-a8e0-f4559325afcd admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:21.639316 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900de98-de3a-4b0f-b51f-421c7f846313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:21.639972 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900de98-de3a-4b0f-b51f-421c7f846313 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:21.640250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900de98-de3a-4b0f-b51f-421c7f846313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:21.640521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900de98-de3a-4b0f-b51f-421c7f846313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:21.641173 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900de98-de3a-4b0f-b51f-421c7f846313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:21.642930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900de98-de3a-4b0f-b51f-421c7f846313 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:21.653573 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1437/2874] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:44:21.657887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42dd2da-45d7-419e-8746-1fa8ac899025 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:21.669011 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42dd2da-45d7-419e-8746-1fa8ac899025 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:21.674426 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf460d79-4de0-4ccd-8ace-5f717661bae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:21.675160 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf460d79-4de0-4ccd-8ace-5f717661bae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:21.675450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf460d79-4de0-4ccd-8ace-5f717661bae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:21.675782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf460d79-4de0-4ccd-8ace-5f717661bae1 admin admin] PATH_INFO: `/v3/projects/093322120a7d40d290a1a0e96259a1f1/users/189bf89cfbc44914b2287bf3fe044168/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:21.680547 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf460d79-4de0-4ccd-8ace-5f717661bae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=093322120a7d40d290a1a0e96259a1f1, user_id=189bf89cfbc44914b2287bf3fe044168, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:21.680764 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf460d79-4de0-4ccd-8ace-5f717661bae1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:21.692594 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1438/2875] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:44:21 2022] PUT /identity/v3/projects/093322120a7d40d290a1a0e96259a1f1/users/189bf89cfbc44914b2287bf3fe044168/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 23 01:44:21.703331 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed2f49-eb86-41d3-98e3-64574dfc085c None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:21.709301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed2f49-eb86-41d3-98e3-64574dfc085c None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:21.709301 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed2f49-eb86-41d3-98e3-64574dfc085c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:21.711503 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:21.734087 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-b2ed2f49-eb86-41d3-98e3-64574dfc085c None None] MFA Rules not processed for user `189bf89cfbc44914b2287bf3fe044168`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:21.735418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b2ed2f49-eb86-41d3-98e3-64574dfc085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:21.807631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1438/2876] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:21.821062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf460d79-4de0-4ccd-8ace-5f717661bae1 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:21.822848 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cf460d79-4de0-4ccd-8ace-5f717661bae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:21.885454 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf460d79-4de0-4ccd-8ace-5f717661bae1 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:21.889588 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a476a-4e29-4be6-9e91-48237fcbf7e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:21.890230 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a476a-4e29-4be6-9e91-48237fcbf7e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:21.890506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a476a-4e29-4be6-9e91-48237fcbf7e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:21.890777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a476a-4e29-4be6-9e91-48237fcbf7e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:21.953246 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a476a-4e29-4be6-9e91-48237fcbf7e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:21.954704 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a476a-4e29-4be6-9e91-48237fcbf7e4 service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:21.964481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1439/2877] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:44:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:44:24.173703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed2f49-eb86-41d3-98e3-64574dfc085c None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:24.175287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-b2ed2f49-eb86-41d3-98e3-64574dfc085c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:24.187713 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed2f49-eb86-41d3-98e3-64574dfc085c None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:24.191344 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769206e3-4831-4d98-b9df-89f8f012780e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:24.191951 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769206e3-4831-4d98-b9df-89f8f012780e service cinder] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:24.192273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769206e3-4831-4d98-b9df-89f8f012780e service cinder] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:24.192589 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769206e3-4831-4d98-b9df-89f8f012780e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:24.255943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769206e3-4831-4d98-b9df-89f8f012780e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:24.257394 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769206e3-4831-4d98-b9df-89f8f012780e service cinder] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:24.268386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1439/2878] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:44:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:44:25.645986 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a476a-4e29-4be6-9e91-48237fcbf7e4 service cinder] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:25.654817 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a476a-4e29-4be6-9e91-48237fcbf7e4 service cinder] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:25.658573 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f2c3ea-7e38-4b51-924d-d91cbe9bc4eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:25.659102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c3ea-7e38-4b51-924d-d91cbe9bc4eb service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:25.659382 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c3ea-7e38-4b51-924d-d91cbe9bc4eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:25.659697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f2c3ea-7e38-4b51-924d-d91cbe9bc4eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:25.669835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2c3ea-7e38-4b51-924d-d91cbe9bc4eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:25.671858 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f2c3ea-7e38-4b51-924d-d91cbe9bc4eb service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:25.681408 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1440/2879] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:44:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:44:30.615353 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769206e3-4831-4d98-b9df-89f8f012780e service cinder] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:30.677348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769206e3-4831-4d98-b9df-89f8f012780e service cinder] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:30.681993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:30.682635 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:30.682918 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:30.683198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] PATH_INFO: `/v3/users/a5ec98a403dc4c459bb66dc4236f871a` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:30.684383 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ec98a403dc4c459bb66dc4236f871a)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:30.686318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:30.777501 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:30.920060 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ec98a403dc4c459bb66dc4236f871a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:30.927427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ec98a403dc4c459bb66dc4236f871a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:30.927427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ec98a403dc4c459bb66dc4236f871a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:30.929306 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ec98a403dc4c459bb66dc4236f871a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:30.942738 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ec98a403dc4c459bb66dc4236f871a'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:31.021724 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1440/2880] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:44:30 2022] DELETE /identity/v3/users/a5ec98a403dc4c459bb66dc4236f871a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:31.026743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea8415-6f67-403d-a672-60178f233626 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:31.027221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea8415-6f67-403d-a672-60178f233626 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:31.027324 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea8415-6f67-403d-a672-60178f233626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:31.047271 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-e5ea8415-6f67-403d-a672-60178f233626 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:31.048525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e5ea8415-6f67-403d-a672-60178f233626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:31.089049 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:31.108856 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1441/2881] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:31.118888 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:31.120370 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:31.179712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b880c48-0340-4c36-8c52-e8f2d84ee655 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:31.183449 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee89d3-c14f-45b7-8415-a355e8beff84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:31.184100 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee89d3-c14f-45b7-8415-a355e8beff84 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:31.184377 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee89d3-c14f-45b7-8415-a355e8beff84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:31.184656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee89d3-c14f-45b7-8415-a355e8beff84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:31.194623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee89d3-c14f-45b7-8415-a355e8beff84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:31.196097 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee89d3-c14f-45b7-8415-a355e8beff84 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:31.205853 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1441/2882] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:44:31.670413 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea8415-6f67-403d-a672-60178f233626 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:31.671202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e5ea8415-6f67-403d-a672-60178f233626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:31.684777 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea8415-6f67-403d-a672-60178f233626 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:31.689533 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:31.690177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:31.690461 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:31.690741 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] PATH_INFO: `/v3/projects/7f96e3362a23448394155a97547f9b90` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:31.691910 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f96e3362a23448394155a97547f9b90)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:31.693867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:31.841506 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f96e3362a23448394155a97547f9b90', 'request_id': 'req-b4a9e599-baa4-4021-a365-e208062f5499'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:31.847349 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f96e3362a23448394155a97547f9b90', 'request_id': 'req-b4a9e599-baa4-4021-a365-e208062f5499'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:31.923887 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f96e3362a23448394155a97547f9b90', 'request_id': 'req-b4a9e599-baa4-4021-a365-e208062f5499'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:31.936718 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] The token cache is being invalidate because project 7f96e3362a23448394155a97547f9b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:44:31.937005 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:31.937648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:31.939131 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1442/2883] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:44:31 2022] DELETE /identity/v3/projects/7f96e3362a23448394155a97547f9b90 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:31.943335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee89d3-c14f-45b7-8415-a355e8beff84 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:31.954304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee89d3-c14f-45b7-8415-a355e8beff84 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:32.011712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:32.012346 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:32.012633 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:32.012914 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] PATH_INFO: `/v3/users/1bebc347aede4cf6840423e596f165d7` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:32.014056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bebc347aede4cf6840423e596f165d7)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:32.016685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:32.179939 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bebc347aede4cf6840423e596f165d7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:32.180517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bebc347aede4cf6840423e596f165d7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:32.181193 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bebc347aede4cf6840423e596f165d7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:32.183374 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:32.189161 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bebc347aede4cf6840423e596f165d7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:32.201610 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bebc347aede4cf6840423e596f165d7'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:32.277782 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1442/2884] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:44:31 2022] DELETE /identity/v3/users/1bebc347aede4cf6840423e596f165d7 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:32.734604 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:32.736199 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:32.750652 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a9e599-baa4-4021-a365-e208062f5499 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:32.814121 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:32.814931 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:32.815216 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:32.815492 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] PATH_INFO: `/v3/projects/4f1439bbfd7d4ac5aefe7c595d8eed6e` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:32.816697 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f1439bbfd7d4ac5aefe7c595d8eed6e)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:32.818601 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:32.933818 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f1439bbfd7d4ac5aefe7c595d8eed6e', 'request_id': 'req-ce56763a-c33b-4c2c-bf99-adacd54205c2'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:32.940330 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f1439bbfd7d4ac5aefe7c595d8eed6e', 'request_id': 'req-ce56763a-c33b-4c2c-bf99-adacd54205c2'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:33.030846 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f1439bbfd7d4ac5aefe7c595d8eed6e', 'request_id': 'req-ce56763a-c33b-4c2c-bf99-adacd54205c2'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:33.040014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:33.044452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] The token cache is being invalidate because project 4f1439bbfd7d4ac5aefe7c595d8eed6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:44:33.044743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:33.045425 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:33.047502 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1443/2885] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:44:32 2022] DELETE /identity/v3/projects/4f1439bbfd7d4ac5aefe7c595d8eed6e => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:33.051814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:33.053305 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:33.066656 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2704a31e-b4c7-4545-b31c-1020eb0fe64c admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:33.122989 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:33.123712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:33.124479 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:33.124755 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] PATH_INFO: `/v3/users/189bf89cfbc44914b2287bf3fe044168` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:33.126091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=189bf89cfbc44914b2287bf3fe044168)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:33.128177 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:33.311023 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '189bf89cfbc44914b2287bf3fe044168'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:33.311752 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '189bf89cfbc44914b2287bf3fe044168'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:33.312401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '189bf89cfbc44914b2287bf3fe044168'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:33.320113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '189bf89cfbc44914b2287bf3fe044168'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:33.332796 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-480f8157-b3fe-47eb-84a5-7226550017b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '189bf89cfbc44914b2287bf3fe044168'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:33.397977 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1443/2886] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:44:33 2022] DELETE /identity/v3/users/189bf89cfbc44914b2287bf3fe044168 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:33.806070 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:33.807790 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:33.822074 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56763a-c33b-4c2c-bf99-adacd54205c2 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:33.892248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:33.892909 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:33.893189 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:33.893464 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] PATH_INFO: `/v3/projects/093322120a7d40d290a1a0e96259a1f1` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:33.894762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=093322120a7d40d290a1a0e96259a1f1)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:33.896829 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:34.023678 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093322120a7d40d290a1a0e96259a1f1', 'request_id': 'req-dd202a33-b715-4a1e-874f-1d8e488349b1'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:34.031444 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093322120a7d40d290a1a0e96259a1f1', 'request_id': 'req-dd202a33-b715-4a1e-874f-1d8e488349b1'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:34.090398 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093322120a7d40d290a1a0e96259a1f1', 'request_id': 'req-dd202a33-b715-4a1e-874f-1d8e488349b1'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:34.102786 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] The token cache is being invalidate because project 093322120a7d40d290a1a0e96259a1f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:44:34.103098 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:34.103779 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-dd202a33-b715-4a1e-874f-1d8e488349b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:44:34.105260 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1444/2887] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:44:33 2022] DELETE /identity/v3/projects/093322120a7d40d290a1a0e96259a1f1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:34.111249 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fad22-06bd-4aec-bc84-e52aaf0751c4 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:34.111892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fad22-06bd-4aec-bc84-e52aaf0751c4 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:34.111892 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3fad22-06bd-4aec-bc84-e52aaf0751c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:34.132936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-1a3fad22-06bd-4aec-bc84-e52aaf0751c4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:34.134101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-1a3fad22-06bd-4aec-bc84-e52aaf0751c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:34.182456 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:34.200728 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1444/2888] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:44:34.209525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf109e56-dacf-45df-a7e9-6147b182a063 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:34.209811 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf109e56-dacf-45df-a7e9-6147b182a063 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:34.210113 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf109e56-dacf-45df-a7e9-6147b182a063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:34.231228 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-bf109e56-dacf-45df-a7e9-6147b182a063 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:34.235618 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-bf109e56-dacf-45df-a7e9-6147b182a063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:34.240762 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1445/2889] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:34.245491 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fad22-06bd-4aec-bc84-e52aaf0751c4 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:34.256155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3fad22-06bd-4aec-bc84-e52aaf0751c4 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:34.262366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0f97f-251f-4f78-be46-68771ea03306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:34.263002 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f97f-251f-4f78-be46-68771ea03306 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:34.263280 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f97f-251f-4f78-be46-68771ea03306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:34.263653 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0f97f-251f-4f78-be46-68771ea03306 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:34.264352 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0f97f-251f-4f78-be46-68771ea03306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:44:34.270000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0f97f-251f-4f78-be46-68771ea03306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:34.270000 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0f97f-251f-4f78-be46-68771ea03306 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:34.278930 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1445/2890] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:44:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 23 01:44:34.284102 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf109e56-dacf-45df-a7e9-6147b182a063 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:34.294096 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf109e56-dacf-45df-a7e9-6147b182a063 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:34.299020 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7bf8b-3482-4838-b663-2f73e583c22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:34.299663 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7bf8b-3482-4838-b663-2f73e583c22a admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:34.299941 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7bf8b-3482-4838-b663-2f73e583c22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:34.300295 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7bf8b-3482-4838-b663-2f73e583c22a admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:34.300861 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7bf8b-3482-4838-b663-2f73e583c22a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:34.302768 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7bf8b-3482-4838-b663-2f73e583c22a admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:34.392348 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1446/2891] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:44:34 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 23 01:44:34.396749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0f97f-251f-4f78-be46-68771ea03306 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:34.408113 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0f97f-251f-4f78-be46-68771ea03306 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:34.466012 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fe993-b61c-47cc-85c7-9016e7e04a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:34.466623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fe993-b61c-47cc-85c7-9016e7e04a5a admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:34.466898 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fe993-b61c-47cc-85c7-9016e7e04a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:34.467167 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fe993-b61c-47cc-85c7-9016e7e04a5a admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:34.468506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fe993-b61c-47cc-85c7-9016e7e04a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:34.470419 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fe993-b61c-47cc-85c7-9016e7e04a5a admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:34.551673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1446/2892] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:44:34 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:44:34.556017 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7bf8b-3482-4838-b663-2f73e583c22a admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:34.567221 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7bf8b-3482-4838-b663-2f73e583c22a admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:34.571893 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98a7f3-8bcc-4efe-9375-028ec120356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:34.572487 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98a7f3-8bcc-4efe-9375-028ec120356e admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:34.572761 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98a7f3-8bcc-4efe-9375-028ec120356e admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:34.573043 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98a7f3-8bcc-4efe-9375-028ec120356e admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:34.573675 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98a7f3-8bcc-4efe-9375-028ec120356e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:34.575287 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d98a7f3-8bcc-4efe-9375-028ec120356e admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:34.585740 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1447/2893] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:44:34.592227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fe993-b61c-47cc-85c7-9016e7e04a5a admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:34.603427 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fe993-b61c-47cc-85c7-9016e7e04a5a admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:34.608654 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be14fc-6927-44cd-9b98-0d085a973236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:34.609262 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be14fc-6927-44cd-9b98-0d085a973236 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:34.610070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be14fc-6927-44cd-9b98-0d085a973236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:34.610070 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be14fc-6927-44cd-9b98-0d085a973236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:34.610433 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be14fc-6927-44cd-9b98-0d085a973236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:34.612055 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be14fc-6927-44cd-9b98-0d085a973236 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:34.623363 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1447/2894] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:44:34.632536 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98a7f3-8bcc-4efe-9375-028ec120356e admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:34.642922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d98a7f3-8bcc-4efe-9375-028ec120356e admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:34.647605 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729a7b0-86b1-42b6-a097-5fa7489d3e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:34.652512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729a7b0-86b1-42b6-a097-5fa7489d3e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:34.652512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729a7b0-86b1-42b6-a097-5fa7489d3e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:34.652512 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729a7b0-86b1-42b6-a097-5fa7489d3e97 admin admin] PATH_INFO: `/v3/projects/ad40312040a946fe9d0daadb5b65093f/users/1887ca4cca6b477d880cdd240eb8bc38/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:34.663305 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729a7b0-86b1-42b6-a097-5fa7489d3e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad40312040a946fe9d0daadb5b65093f, user_id=1887ca4cca6b477d880cdd240eb8bc38, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:34.666052 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729a7b0-86b1-42b6-a097-5fa7489d3e97 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:34.756940 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1448/2895] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:44:34 2022] PUT /identity/v3/projects/ad40312040a946fe9d0daadb5b65093f/users/1887ca4cca6b477d880cdd240eb8bc38/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:44:34.769208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82195fc0-a2d6-4d0f-a015-c1a54aaad7ed None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:34.769489 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82195fc0-a2d6-4d0f-a015-c1a54aaad7ed None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:34.769770 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82195fc0-a2d6-4d0f-a015-c1a54aaad7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:34.799802 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-82195fc0-a2d6-4d0f-a015-c1a54aaad7ed None None] MFA Rules not processed for user `1887ca4cca6b477d880cdd240eb8bc38`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:44:34.801059 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-82195fc0-a2d6-4d0f-a015-c1a54aaad7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:34.871349 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1448/2896] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:44:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:44:34.884116 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729a7b0-86b1-42b6-a097-5fa7489d3e97 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:44:34.885895 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d729a7b0-86b1-42b6-a097-5fa7489d3e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:44:34.915622 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:34.947430 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729a7b0-86b1-42b6-a097-5fa7489d3e97 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:44:34.951802 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcd702-e674-42c6-b9ba-49de5e7831fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51ae9d865bb4442490ca61b18d1f7ec7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:44:34.952428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcd702-e674-42c6-b9ba-49de5e7831fd service cinder] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:44:34.952705 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcd702-e674-42c6-b9ba-49de5e7831fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:44:34.952980 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcd702-e674-42c6-b9ba-49de5e7831fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:44:34.962033 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcd702-e674-42c6-b9ba-49de5e7831fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:44:34.963481 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcd702-e674-42c6-b9ba-49de5e7831fd service cinder] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:44:34.972914 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1449/2897] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:44:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:44:44.868234 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:44:45.015334 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=15 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:45:14.635020 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82195fc0-a2d6-4d0f-a015-c1a54aaad7ed None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:14.701990 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82195fc0-a2d6-4d0f-a015-c1a54aaad7ed None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:14.707445 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:14.708118 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:14.708401 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:14.708676 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] PATH_INFO: `/v3/users/1887ca4cca6b477d880cdd240eb8bc38` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:14.709832 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1887ca4cca6b477d880cdd240eb8bc38)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:14.711833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:14.913276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1887ca4cca6b477d880cdd240eb8bc38'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:45:14.914034 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1887ca4cca6b477d880cdd240eb8bc38'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:45:14.914729 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1887ca4cca6b477d880cdd240eb8bc38'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:45:14.922564 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1887ca4cca6b477d880cdd240eb8bc38'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:45:14.935579 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1887ca4cca6b477d880cdd240eb8bc38'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:45:15.037608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1449/2898] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:45:14 2022] DELETE /identity/v3/users/1887ca4cca6b477d880cdd240eb8bc38 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:45:15.042372 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dff05-2507-4685-863c-1403e026f7a4 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:15.043208 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dff05-2507-4685-863c-1403e026f7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:15.043410 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7dff05-2507-4685-863c-1403e026f7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:15.063609 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-2b7dff05-2507-4685-863c-1403e026f7a4 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:45:15.065014 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2b7dff05-2507-4685-863c-1403e026f7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:15.130062 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1450/2899] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:45:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:45:15.140673 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:15.142146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:15.211993 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfc2e0-efe8-4db6-8d38-5496e88040e9 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:15.215925 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dbe88-0c18-43e2-8e3d-94889ed1d6c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:15.216549 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dbe88-0c18-43e2-8e3d-94889ed1d6c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:15.216835 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dbe88-0c18-43e2-8e3d-94889ed1d6c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:15.217115 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dbe88-0c18-43e2-8e3d-94889ed1d6c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:15.227840 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dbe88-0c18-43e2-8e3d-94889ed1d6c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:15.229300 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dbe88-0c18-43e2-8e3d-94889ed1d6c0 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:15.240366 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1450/2900] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:45:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:45:15.782590 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7dff05-2507-4685-863c-1403e026f7a4 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:15.784443 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-2b7dff05-2507-4685-863c-1403e026f7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:15.803529 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7dff05-2507-4685-863c-1403e026f7a4 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:15.811939 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:15.813047 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:15.813525 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:15.814042 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] PATH_INFO: `/v3/projects/ad40312040a946fe9d0daadb5b65093f` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:15.816700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad40312040a946fe9d0daadb5b65093f)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:15.820184 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:15.957808 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad40312040a946fe9d0daadb5b65093f', 'request_id': 'req-aa6af531-a19e-44bd-949b-4222e01fdeef'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:45:15.963855 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad40312040a946fe9d0daadb5b65093f', 'request_id': 'req-aa6af531-a19e-44bd-949b-4222e01fdeef'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:45:16.035509 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad40312040a946fe9d0daadb5b65093f', 'request_id': 'req-aa6af531-a19e-44bd-949b-4222e01fdeef'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:45:16.048991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] The token cache is being invalidate because project ad40312040a946fe9d0daadb5b65093f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:45:16.049689 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:45:16.050029 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-aa6af531-a19e-44bd-949b-4222e01fdeef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:45:16.051721 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1451/2901] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:45:15 2022] DELETE /identity/v3/projects/ad40312040a946fe9d0daadb5b65093f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:45:16.056795 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3da98-b576-49a9-a2d6-d2c8e31562d6 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:16.057348 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3da98-b576-49a9-a2d6-d2c8e31562d6 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:16.057479 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3da98-b576-49a9-a2d6-d2c8e31562d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:16.083594 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-dbd3da98-b576-49a9-a2d6-d2c8e31562d6 None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:45:16.085945 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-dbd3da98-b576-49a9-a2d6-d2c8e31562d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:16.105811 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:45:16.156908 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1451/2902] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:45:16.166323 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff76116-85ba-4965-98ff-cd6d380ac4fb None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:16.166619 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff76116-85ba-4965-98ff-cd6d380ac4fb None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:16.166871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff76116-85ba-4965-98ff-cd6d380ac4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:16.188813 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-cff76116-85ba-4965-98ff-cd6d380ac4fb None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:45:16.190046 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-cff76116-85ba-4965-98ff-cd6d380ac4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:16.198824 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1452/2903] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:45:16.203356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3da98-b576-49a9-a2d6-d2c8e31562d6 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:16.214364 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3da98-b576-49a9-a2d6-d2c8e31562d6 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:16.219252 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8dd8f-e23e-4cbd-8421-c9b4f32699e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:16.219908 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8dd8f-e23e-4cbd-8421-c9b4f32699e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:16.220190 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8dd8f-e23e-4cbd-8421-c9b4f32699e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:16.220467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8dd8f-e23e-4cbd-8421-c9b4f32699e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:16.221130 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8dd8f-e23e-4cbd-8421-c9b4f32699e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81960) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 23 01:45:16.221438 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8dd8f-e23e-4cbd-8421-c9b4f32699e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:16.223052 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8dd8f-e23e-4cbd-8421-c9b4f32699e8 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:16.235740 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1452/2904] 10.222.0.64 () {58 vars in 1153 bytes} [Fri Sep 23 01:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 23 01:45:16.241461 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff76116-85ba-4965-98ff-cd6d380ac4fb None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:16.256265 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff76116-85ba-4965-98ff-cd6d380ac4fb None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:16.259136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61634ec-e2cf-4762-91d8-9481da20d4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:16.259781 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61634ec-e2cf-4762-91d8-9481da20d4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:16.260064 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61634ec-e2cf-4762-91d8-9481da20d4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:16.260340 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61634ec-e2cf-4762-91d8-9481da20d4c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:16.260979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61634ec-e2cf-4762-91d8-9481da20d4c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:16.262979 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61634ec-e2cf-4762-91d8-9481da20d4c7 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:16.376343 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1453/2905] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:45:16 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:45:16.381874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8dd8f-e23e-4cbd-8421-c9b4f32699e8 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:16.397602 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8dd8f-e23e-4cbd-8421-c9b4f32699e8 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:16.470608 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13033f29-4a41-44c5-9bb8-042a8add20ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:16.471209 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13033f29-4a41-44c5-9bb8-042a8add20ea admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:16.471425 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13033f29-4a41-44c5-9bb8-042a8add20ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:16.472222 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13033f29-4a41-44c5-9bb8-042a8add20ea admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:16.472386 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13033f29-4a41-44c5-9bb8-042a8add20ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:16.474355 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13033f29-4a41-44c5-9bb8-042a8add20ea admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:16.575461 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1453/2906] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:45:16 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:45:16.583156 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61634ec-e2cf-4762-91d8-9481da20d4c7 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:16.594117 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61634ec-e2cf-4762-91d8-9481da20d4c7 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:16.599547 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07348776-fc42-4cd1-b0e0-c98a28acf7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:16.600205 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07348776-fc42-4cd1-b0e0-c98a28acf7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:16.600490 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07348776-fc42-4cd1-b0e0-c98a28acf7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:16.600765 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07348776-fc42-4cd1-b0e0-c98a28acf7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:16.601743 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07348776-fc42-4cd1-b0e0-c98a28acf7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:16.603445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07348776-fc42-4cd1-b0e0-c98a28acf7b1 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:16.619690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1454/2907] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:45:16.623936 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13033f29-4a41-44c5-9bb8-042a8add20ea admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:16.635239 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13033f29-4a41-44c5-9bb8-042a8add20ea admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:16.640431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eedc8b-2c42-4c57-92be-a45b11aba42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:16.641056 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eedc8b-2c42-4c57-92be-a45b11aba42d admin admin] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:16.641343 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eedc8b-2c42-4c57-92be-a45b11aba42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:16.641623 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eedc8b-2c42-4c57-92be-a45b11aba42d admin admin] PATH_INFO: `/v3/roles` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:16.642297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eedc8b-2c42-4c57-92be-a45b11aba42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:16.644031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eedc8b-2c42-4c57-92be-a45b11aba42d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:16.656329 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1454/2908] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:45:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:45:16.664707 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07348776-fc42-4cd1-b0e0-c98a28acf7b1 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:16.677224 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07348776-fc42-4cd1-b0e0-c98a28acf7b1 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:16.682384 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5335f-83cc-47cc-b46b-901b8347330b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:16.683083 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5335f-83cc-47cc-b46b-901b8347330b admin admin] REQUEST_METHOD: `PUT` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:16.683467 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5335f-83cc-47cc-b46b-901b8347330b admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:16.683683 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5335f-83cc-47cc-b46b-901b8347330b admin admin] PATH_INFO: `/v3/projects/403b0f1ad71c459581759568e4fcd1ca/users/0857c0af9c944a2fb6e75c7297945fc0/roles/e5a65ae2d0884e419cdbc70c9223eac2` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:16.702143 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5335f-83cc-47cc-b46b-901b8347330b admin admin] RBAC: Authorizing `identity:create_grant(project_id=403b0f1ad71c459581759568e4fcd1ca, user_id=0857c0af9c944a2fb6e75c7297945fc0, role_id=e5a65ae2d0884e419cdbc70c9223eac2)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:16.704982 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5335f-83cc-47cc-b46b-901b8347330b admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:16.793164 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1455/2909] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:45:16 2022] PUT /identity/v3/projects/403b0f1ad71c459581759568e4fcd1ca/users/0857c0af9c944a2fb6e75c7297945fc0/roles/e5a65ae2d0884e419cdbc70c9223eac2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:45:16.798757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bac226-3be7-40d5-849b-4e4125620e6d None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:16.799128 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bac226-3be7-40d5-849b-4e4125620e6d None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:16.799418 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bac226-3be7-40d5-849b-4e4125620e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:16.821964 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-f6bac226-3be7-40d5-849b-4e4125620e6d None None] MFA Rules not processed for user `efae8da899ef465d9872282b2bb91fef`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:45:16.823264 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-f6bac226-3be7-40d5-849b-4e4125620e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:16.904949 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1455/2910] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:45:16.920016 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5335f-83cc-47cc-b46b-901b8347330b admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:16.921922 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-e9d5335f-83cc-47cc-b46b-901b8347330b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:16.951307 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:45:16.997194 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5335f-83cc-47cc-b46b-901b8347330b admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:17.001041 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341c652-9411-45f2-bdff-44561a1661b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:17.001835 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341c652-9411-45f2-bdff-44561a1661b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:17.002105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341c652-9411-45f2-bdff-44561a1661b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:17.002381 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341c652-9411-45f2-bdff-44561a1661b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:17.012998 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341c652-9411-45f2-bdff-44561a1661b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:17.014728 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341c652-9411-45f2-bdff-44561a1661b5 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:17.026091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1456/2911] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:45:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:45:23.364646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b6c0d-2fd5-4398-a8a0-6815b9de6ba3 None None] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:23.365287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b6c0d-2fd5-4398-a8a0-6815b9de6ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:23.365287 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0b6c0d-2fd5-4398-a8a0-6815b9de6ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:23.403827 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.auth.core [None req-3c0b6c0d-2fd5-4398-a8a0-6815b9de6ba3 None None] MFA Rules not processed for user `0857c0af9c944a2fb6e75c7297945fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81960) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:45:23.405171 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-3c0b6c0d-2fd5-4398-a8a0-6815b9de6ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:23.428006 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:45:23.512325 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1456/2912] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:45:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 23 01:45:23.518889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341c652-9411-45f2-bdff-44561a1661b5 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:23.531667 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341c652-9411-45f2-bdff-44561a1661b5 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:23.544411 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b5791-c2d8-4ac3-b4bd-e3fbd435d5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:23.545298 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5791-c2d8-4ac3-b4bd-e3fbd435d5ac admin admin] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:23.545653 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5791-c2d8-4ac3-b4bd-e3fbd435d5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:23.546105 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b5791-c2d8-4ac3-b4bd-e3fbd435d5ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:23.546970 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b5791-c2d8-4ac3-b4bd-e3fbd435d5ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:23.549875 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b5791-c2d8-4ac3-b4bd-e3fbd435d5ac admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:23.690700 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1457/2913] 10.222.0.64 () {60 vars in 1153 bytes} [Fri Sep 23 01:45:23 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:45:23.695886 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b6c0d-2fd5-4398-a8a0-6815b9de6ba3 None None] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:23.709390 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0b6c0d-2fd5-4398-a8a0-6815b9de6ba3 None None] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:23.789032 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c879f6-c5e0-413b-9947-d6e769d90d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:23.789677 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c879f6-c5e0-413b-9947-d6e769d90d92 admin admin] REQUEST_METHOD: `POST` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:23.789963 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c879f6-c5e0-413b-9947-d6e769d90d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:23.790244 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c879f6-c5e0-413b-9947-d6e769d90d92 admin admin] PATH_INFO: `/v3/users` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:23.790912 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c879f6-c5e0-413b-9947-d6e769d90d92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:23.792979 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c879f6-c5e0-413b-9947-d6e769d90d92 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:23.922151 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1457/2914] 10.222.0.64 () {60 vars in 1144 bytes} [Fri Sep 23 01:45:23 2022] POST /identity/v3/users => generated 298 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 23 01:45:23.927204 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b5791-c2d8-4ac3-b4bd-e3fbd435d5ac admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:23.940008 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b5791-c2d8-4ac3-b4bd-e3fbd435d5ac admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:23.947883 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718eda2-f1fd-4cec-b715-d3f341ad7244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:23.948515 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718eda2-f1fd-4cec-b715-d3f341ad7244 admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:23.948892 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718eda2-f1fd-4cec-b715-d3f341ad7244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:23.949177 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718eda2-f1fd-4cec-b715-d3f341ad7244 admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:23.949864 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718eda2-f1fd-4cec-b715-d3f341ad7244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:23.951908 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718eda2-f1fd-4cec-b715-d3f341ad7244 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:23.967954 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1458/2915] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 23 01:45:23.973188 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c879f6-c5e0-413b-9947-d6e769d90d92 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:23.984568 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c879f6-c5e0-413b-9947-d6e769d90d92 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:23.990614 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69d32c-83a0-45fa-b6fb-f4f9af06aa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:23.991431 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69d32c-83a0-45fa-b6fb-f4f9af06aa41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:23.991866 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69d32c-83a0-45fa-b6fb-f4f9af06aa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:23.992958 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69d32c-83a0-45fa-b6fb-f4f9af06aa41 admin admin] PATH_INFO: `/v3/projects/8710c6faa90b498fb6e3bae1a0df6853/users/5da884d486704d019a23d46916723534/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:24.016946 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69d32c-83a0-45fa-b6fb-f4f9af06aa41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8710c6faa90b498fb6e3bae1a0df6853, user_id=5da884d486704d019a23d46916723534, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:24.020518 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d69d32c-83a0-45fa-b6fb-f4f9af06aa41 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:24.100274 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1458/2916] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:45:23 2022] PUT /identity/v3/projects/8710c6faa90b498fb6e3bae1a0df6853/users/5da884d486704d019a23d46916723534/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:45:24.105477 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718eda2-f1fd-4cec-b715-d3f341ad7244 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:24.117452 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718eda2-f1fd-4cec-b715-d3f341ad7244 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:24.199497 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30292b2-b83b-400c-a69a-4d4b53d56eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:24.199871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30292b2-b83b-400c-a69a-4d4b53d56eec admin admin] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:24.200151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30292b2-b83b-400c-a69a-4d4b53d56eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:24.200427 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30292b2-b83b-400c-a69a-4d4b53d56eec admin admin] PATH_INFO: `/v3/roles` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:24.201095 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30292b2-b83b-400c-a69a-4d4b53d56eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:24.202769 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30292b2-b83b-400c-a69a-4d4b53d56eec admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:24.216800 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1459/2917] 10.222.0.64 () {58 vars in 1122 bytes} [Fri Sep 23 01:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 23 01:45:24.221556 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69d32c-83a0-45fa-b6fb-f4f9af06aa41 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:24.233241 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69d32c-83a0-45fa-b6fb-f4f9af06aa41 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:24.238780 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4083b-1614-4e09-a195-e757e12b79c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:24.239470 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4083b-1614-4e09-a195-e757e12b79c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:24.239784 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4083b-1614-4e09-a195-e757e12b79c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:24.240058 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4083b-1614-4e09-a195-e757e12b79c5 admin admin] PATH_INFO: `/v3/projects/8710c6faa90b498fb6e3bae1a0df6853/users/5da884d486704d019a23d46916723534/roles/8928ed5fc98f44048f4c1b90fae793b6` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:24.242149 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4083b-1614-4e09-a195-e757e12b79c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8710c6faa90b498fb6e3bae1a0df6853, user_id=5da884d486704d019a23d46916723534, role_id=8928ed5fc98f44048f4c1b90fae793b6)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:24.245053 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4083b-1614-4e09-a195-e757e12b79c5 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:24.258061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1459/2918] 10.222.0.64 () {60 vars in 1483 bytes} [Fri Sep 23 01:45:24 2022] PUT /identity/v3/projects/8710c6faa90b498fb6e3bae1a0df6853/users/5da884d486704d019a23d46916723534/roles/8928ed5fc98f44048f4c1b90fae793b6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:45:31.854277 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9bbd7-5984-491e-831d-4e0999e27f40 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:31.855091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9bbd7-5984-491e-831d-4e0999e27f40 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:31.855091 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9bbd7-5984-491e-831d-4e0999e27f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:31.894937 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-6ac9bbd7-5984-491e-831d-4e0999e27f40 None None] MFA Rules not processed for user `5da884d486704d019a23d46916723534`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:45:31.897247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-6ac9bbd7-5984-491e-831d-4e0999e27f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:32.001153 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1460/2919] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:45:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:45:32.013903 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4083b-1614-4e09-a195-e757e12b79c5 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:32.015521 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-37e4083b-1614-4e09-a195-e757e12b79c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:32.107968 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4083b-1614-4e09-a195-e757e12b79c5 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:32.112335 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42112309-7a82-4360-8cda-0affddb73da5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f10c258fe5f45ccab1669fa32ed6029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:32.112971 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42112309-7a82-4360-8cda-0affddb73da5 service glance] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:32.113261 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42112309-7a82-4360-8cda-0affddb73da5 service glance] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:32.113592 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42112309-7a82-4360-8cda-0affddb73da5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:32.118281 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:45:32.191540 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42112309-7a82-4360-8cda-0affddb73da5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:32.193874 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42112309-7a82-4360-8cda-0affddb73da5 service glance] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:32.205833 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1460/2920] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:45:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:45:32.469575 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601454b1-f32a-4f55-a110-074d69661ffe None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:32.469871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601454b1-f32a-4f55-a110-074d69661ffe None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:32.470151 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601454b1-f32a-4f55-a110-074d69661ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:32.500087 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:45:32.543120 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-601454b1-f32a-4f55-a110-074d69661ffe None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:45:32.544600 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-601454b1-f32a-4f55-a110-074d69661ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:32.624398 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1461/2921] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:45:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:45:32.784297 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42112309-7a82-4360-8cda-0affddb73da5 service glance] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:32.785766 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-42112309-7a82-4360-8cda-0affddb73da5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:32.892356 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42112309-7a82-4360-8cda-0affddb73da5 service glance] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:32.897924 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e82c89-8b07-498e-b9ad-cebd7b9c287f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:32.898561 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e82c89-8b07-498e-b9ad-cebd7b9c287f service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:32.898887 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e82c89-8b07-498e-b9ad-cebd7b9c287f service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:32.899213 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e82c89-8b07-498e-b9ad-cebd7b9c287f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:32.913811 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e82c89-8b07-498e-b9ad-cebd7b9c287f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:32.915703 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e82c89-8b07-498e-b9ad-cebd7b9c287f service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:32.925769 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1461/2922] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:45:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:45:42.621322 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=35 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:45:42.921454 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=66 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:45:44.094392 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d597be7-29be-47d5-aabb-050bc0c67452 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:44.094996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d597be7-29be-47d5-aabb-050bc0c67452 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:44.094996 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d597be7-29be-47d5-aabb-050bc0c67452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:44.118508 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8d597be7-29be-47d5-aabb-050bc0c67452 None None] MFA Rules not processed for user `0857c0af9c944a2fb6e75c7297945fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:45:44.127521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8d597be7-29be-47d5-aabb-050bc0c67452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:44.127521 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1462/2923] 10.222.0.64 () {58 vars in 958 bytes} [Fri Sep 23 01:45:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:45:44.169021 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e82c89-8b07-498e-b9ad-cebd7b9c287f service swift] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:44.179943 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e82c89-8b07-498e-b9ad-cebd7b9c287f service swift] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:44.184398 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4a34e-ece2-41b0-a7be-42f434ce923d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f10c258fe5f45ccab1669fa32ed6029', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:44.185088 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4a34e-ece2-41b0-a7be-42f434ce923d service glance] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:44.185421 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4a34e-ece2-41b0-a7be-42f434ce923d service glance] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:44.185744 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4a34e-ece2-41b0-a7be-42f434ce923d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:44.188044 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-e0d4a34e-ece2-41b0-a7be-42f434ce923d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:44.285572 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4a34e-ece2-41b0-a7be-42f434ce923d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:44.287168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4a34e-ece2-41b0-a7be-42f434ce923d service glance] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:44.300003 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1462/2924] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:45:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:45:44.550660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d597be7-29be-47d5-aabb-050bc0c67452 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:44.550660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8d597be7-29be-47d5-aabb-050bc0c67452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:44.602354 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d597be7-29be-47d5-aabb-050bc0c67452 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:44.605993 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d23ddf-2379-4777-9b10-dc243f3cc4a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:44.613518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d23ddf-2379-4777-9b10-dc243f3cc4a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:44.613518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d23ddf-2379-4777-9b10-dc243f3cc4a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:44.613518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d23ddf-2379-4777-9b10-dc243f3cc4a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:44.613518 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-42d23ddf-2379-4777-9b10-dc243f3cc4a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:44.618956 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d23ddf-2379-4777-9b10-dc243f3cc4a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:44.620428 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d23ddf-2379-4777-9b10-dc243f3cc4a7 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:44.629171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1463/2925] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:45:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:45:45.076091 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4a34e-ece2-41b0-a7be-42f434ce923d service glance] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:45.086327 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4a34e-ece2-41b0-a7be-42f434ce923d service glance] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:45.089942 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25efff-0bee-4b69-91e7-e4fd3fc17851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:45.090560 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25efff-0bee-4b69-91e7-e4fd3fc17851 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:45.090841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25efff-0bee-4b69-91e7-e4fd3fc17851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:45.091112 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25efff-0bee-4b69-91e7-e4fd3fc17851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:45.092864 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ba25efff-0bee-4b69-91e7-e4fd3fc17851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:45.106783 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25efff-0bee-4b69-91e7-e4fd3fc17851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:45.108284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25efff-0bee-4b69-91e7-e4fd3fc17851 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:45.120814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1463/2926] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:45:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:45:48.221239 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222910c-ed73-4b2a-b96b-001e946a84e2 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:48.222066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222910c-ed73-4b2a-b96b-001e946a84e2 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:48.222066 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222910c-ed73-4b2a-b96b-001e946a84e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:48.243809 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-8222910c-ed73-4b2a-b96b-001e946a84e2 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:45:48.245148 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-8222910c-ed73-4b2a-b96b-001e946a84e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:48.254362 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1464/2927] 10.222.0.64 () {58 vars in 1009 bytes} [Fri Sep 23 01:45:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:45:48.273620 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25efff-0bee-4b69-91e7-e4fd3fc17851 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:48.286126 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25efff-0bee-4b69-91e7-e4fd3fc17851 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:48.291838 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d644de-f3e7-4099-a8db-ad57ade586e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:48.292455 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d644de-f3e7-4099-a8db-ad57ade586e9 service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:48.292757 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d644de-f3e7-4099-a8db-ad57ade586e9 service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:48.293011 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d644de-f3e7-4099-a8db-ad57ade586e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:48.302041 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d644de-f3e7-4099-a8db-ad57ade586e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:48.303686 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d644de-f3e7-4099-a8db-ad57ade586e9 service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:48.313106 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1464/2928] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:45:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 23 01:45:56.429202 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222910c-ed73-4b2a-b96b-001e946a84e2 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:56.443361 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222910c-ed73-4b2a-b96b-001e946a84e2 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:56.444883 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d644de-f3e7-4099-a8db-ad57ade586e9 service swift] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:56.447924 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf2772-3562-4ff8-9aba-0395b1908739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:56.451025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf2772-3562-4ff8-9aba-0395b1908739 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:56.451025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf2772-3562-4ff8-9aba-0395b1908739 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:56.451025 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf2772-3562-4ff8-9aba-0395b1908739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:56.453063 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0eaf2772-3562-4ff8-9aba-0395b1908739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:56.456495 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d644de-f3e7-4099-a8db-ad57ade586e9 service swift] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:56.460578 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c1103-326b-4109-9a54-24970ce476ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:56.461202 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c1103-326b-4109-9a54-24970ce476ae service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:56.461485 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c1103-326b-4109-9a54-24970ce476ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:56.461763 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c1103-326b-4109-9a54-24970ce476ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:56.464782 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf2772-3562-4ff8-9aba-0395b1908739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:56.466240 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf2772-3562-4ff8-9aba-0395b1908739 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:56.470133 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c1103-326b-4109-9a54-24970ce476ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:56.472094 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c1103-326b-4109-9a54-24970ce476ae service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:56.475826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1465/2929] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:45:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:45:56.481382 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1465/2930] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:45:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 23 01:45:57.414963 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf2772-3562-4ff8-9aba-0395b1908739 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:45:57.416445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-0eaf2772-3562-4ff8-9aba-0395b1908739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:57.432530 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf2772-3562-4ff8-9aba-0395b1908739 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:45:57.438390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ee27-392c-40a7-9951-ff1c7a669e8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:45:57.439233 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ee27-392c-40a7-9951-ff1c7a669e8a service nova] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:45:57.440248 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ee27-392c-40a7-9951-ff1c7a669e8a service nova] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:45:57.440722 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ee27-392c-40a7-9951-ff1c7a669e8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:45:57.442559 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-a316ee27-392c-40a7-9951-ff1c7a669e8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:45:57.456712 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316ee27-392c-40a7-9951-ff1c7a669e8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:45:57.458048 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316ee27-392c-40a7-9951-ff1c7a669e8a service nova] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:45:57.470207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1466/2931] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:45:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:46:06.477372 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=52 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:46:06.600371 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c1103-326b-4109-9a54-24970ce476ae service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:46:06.604155 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-0f0c1103-326b-4109-9a54-24970ce476ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:46:06.620229 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c1103-326b-4109-9a54-24970ce476ae service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:46:06.626557 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9510f9-a50e-4c95-9c76-f8a82acea9a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c4da5b2130d4fa990adb72f0f6ec2fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:46:06.627200 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9510f9-a50e-4c95-9c76-f8a82acea9a9 service nova] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:46:06.627483 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9510f9-a50e-4c95-9c76-f8a82acea9a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:46:06.627882 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9510f9-a50e-4c95-9c76-f8a82acea9a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:46:06.630402 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-4b9510f9-a50e-4c95-9c76-f8a82acea9a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:46:06.643490 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9510f9-a50e-4c95-9c76-f8a82acea9a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:46:06.645205 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9510f9-a50e-4c95-9c76-f8a82acea9a9 service nova] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:46:06.657439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1466/2932] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:46:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:46:07.464770 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=44 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:46:14.016745 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ee27-392c-40a7-9951-ff1c7a669e8a service nova] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:46:14.030003 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ee27-392c-40a7-9951-ff1c7a669e8a service nova] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:46:14.033690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665ebec-9fc7-402e-9aba-e44ea9ae4fc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:46:14.034285 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ebec-9fc7-402e-9aba-e44ea9ae4fc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:46:14.034566 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ebec-9fc7-402e-9aba-e44ea9ae4fc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:46:14.034836 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ebec-9fc7-402e-9aba-e44ea9ae4fc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:46:14.036690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9665ebec-9fc7-402e-9aba-e44ea9ae4fc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:46:14.156991 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665ebec-9fc7-402e-9aba-e44ea9ae4fc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:46:14.158661 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665ebec-9fc7-402e-9aba-e44ea9ae4fc1 service neutron] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:46:14.180218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1467/2933] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:46:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:46:16.652061 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=6 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:46:18.313132 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9510f9-a50e-4c95-9c76-f8a82acea9a9 service nova] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:46:18.354273 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9510f9-a50e-4c95-9c76-f8a82acea9a9 service nova] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:46:18.358970 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34f2b6-e9d9-4396-abfc-479f85794bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b0a7b4f607d4753bf142e1420bcd987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:46:18.359734 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34f2b6-e9d9-4396-abfc-479f85794bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:46:18.377120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34f2b6-e9d9-4396-abfc-479f85794bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:46:18.377120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34f2b6-e9d9-4396-abfc-479f85794bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:46:18.377120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34f2b6-e9d9-4396-abfc-479f85794bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:46:18.377120 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34f2b6-e9d9-4396-abfc-479f85794bb2 service neutron] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:46:18.384101 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1467/2934] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:46:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:46:22.220254 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665ebec-9fc7-402e-9aba-e44ea9ae4fc1 service neutron] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:46:22.221785 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-9665ebec-9fc7-402e-9aba-e44ea9ae4fc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:46:22.234877 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665ebec-9fc7-402e-9aba-e44ea9ae4fc1 service neutron] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:46:22.239687 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:46:22.240319 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:46:22.240603 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:46:22.240871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] PATH_INFO: `/v3/users/ad7c407cfe924b7a8652f8387c7f77b4` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:46:22.242503 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad7c407cfe924b7a8652f8387c7f77b4)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:46:22.244462 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:46:22.582517 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad7c407cfe924b7a8652f8387c7f77b4'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:22.680690 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad7c407cfe924b7a8652f8387c7f77b4'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:22.681424 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad7c407cfe924b7a8652f8387c7f77b4'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:22.694889 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad7c407cfe924b7a8652f8387c7f77b4'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:22.695560 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad7c407cfe924b7a8652f8387c7f77b4'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:22.708316 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1468/2935] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:46:22 2022] DELETE /identity/v3/users/ad7c407cfe924b7a8652f8387c7f77b4 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:46:22.996712 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34f2b6-e9d9-4396-abfc-479f85794bb2 service neutron] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:46:23.008897 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34f2b6-e9d9-4396-abfc-479f85794bb2 service neutron] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:46:23.070585 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:46:23.071318 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:46:23.071524 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:46:23.072482 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] PATH_INFO: `/v3/projects/f5c5ef59b1b44d8a916be97d8d11493f` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:46:23.073031 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5c5ef59b1b44d8a916be97d8d11493f)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:46:23.075065 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:46:23.217526 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c5ef59b1b44d8a916be97d8d11493f', 'request_id': 'req-dcb18cfb-13ac-428d-989c-ec38fa1ff418'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:23.231778 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c5ef59b1b44d8a916be97d8d11493f', 'request_id': 'req-dcb18cfb-13ac-428d-989c-ec38fa1ff418'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:23.344184 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c5ef59b1b44d8a916be97d8d11493f', 'request_id': 'req-dcb18cfb-13ac-428d-989c-ec38fa1ff418'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:23.356284 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] The token cache is being invalidate because project f5c5ef59b1b44d8a916be97d8d11493f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:46:23.356962 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:23.357238 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:23.358735 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1468/2936] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:46:22 2022] DELETE /identity/v3/projects/f5c5ef59b1b44d8a916be97d8d11493f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:46:23.362949 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:46:23.364450 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:46:23.374942 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:46:23.378370 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e731a1-924f-4e55-aec3-92c0b27c04bc admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:46:23.439473 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:46:23.440136 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:46:23.440415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:46:23.440688 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] PATH_INFO: `/v3/users/33188bf83b6845faa94668978f911a5e` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:46:23.441822 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=33188bf83b6845faa94668978f911a5e)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:46:23.443778 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:46:23.678715 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33188bf83b6845faa94668978f911a5e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:23.824236 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33188bf83b6845faa94668978f911a5e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:23.824867 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33188bf83b6845faa94668978f911a5e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:23.837968 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33188bf83b6845faa94668978f911a5e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:23.838677 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-393ed400-8808-413c-bcdf-e504fcc1c2cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33188bf83b6845faa94668978f911a5e'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:23.859971 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1469/2937] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:46:23 2022] DELETE /identity/v3/users/33188bf83b6845faa94668978f911a5e => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:46:24.140224 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:46:24.141803 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:46:24.157212 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb18cfb-13ac-428d-989c-ec38fa1ff418 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:46:24.210685 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:46:24.221565 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:46:24.222227 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:46:24.223204 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:46:24.223631 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] PATH_INFO: `/v3/projects/6f2ee0c5460b4ca0ad29c419dab374a0` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:46:24.224715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2ee0c5460b4ca0ad29c419dab374a0)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:46:24.227915 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:46:24.373722 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2ee0c5460b4ca0ad29c419dab374a0', 'request_id': 'req-8d5c7832-7aaa-411c-9eab-183e5198b265'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:24.385111 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2ee0c5460b4ca0ad29c419dab374a0', 'request_id': 'req-8d5c7832-7aaa-411c-9eab-183e5198b265'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:24.510481 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2ee0c5460b4ca0ad29c419dab374a0', 'request_id': 'req-8d5c7832-7aaa-411c-9eab-183e5198b265'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:24.523906 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] The token cache is being invalidate because project 6f2ee0c5460b4ca0ad29c419dab374a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81960) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:46:24.524577 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:24.525250 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:46:24.526947 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1469/2938] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:46:24 2022] DELETE /identity/v3/projects/6f2ee0c5460b4ca0ad29c419dab374a0 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:46:33.960466 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=51,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:46:35.081678 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:47:36.019051 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039c14c-be51-4983-aee5-b44f587cb9b3 None None] REQUEST_METHOD: `POST` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:47:36.024218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039c14c-be51-4983-aee5-b44f587cb9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:47:36.024218 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4039c14c-be51-4983-aee5-b44f587cb9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:47:36.049124 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.auth.core [None req-4039c14c-be51-4983-aee5-b44f587cb9b3 None None] MFA Rules not processed for user `fa998fcd51b3404aab5412323f86908f`. Rule list: `[]` (Enabled: `True`). {{(pid=81961) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 23 01:47:36.050355 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4039c14c-be51-4983-aee5-b44f587cb9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:47:36.111691 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1470/2939] 10.222.0.64 () {58 vars in 967 bytes} [Fri Sep 23 01:47:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 23 01:47:36.156700 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:47:36.169814 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:47:36.230956 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c7832-7aaa-411c-9eab-183e5198b265 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:47:36.237095 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39123aed-e375-49b2-bfb8-97693ad8e5c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5a9f3eca3e34fffa7b218064e831d7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:47:36.237689 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39123aed-e375-49b2-bfb8-97693ad8e5c4 service swift] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:47:36.237996 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39123aed-e375-49b2-bfb8-97693ad8e5c4 service swift] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:47:36.238312 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39123aed-e375-49b2-bfb8-97693ad8e5c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:47:36.248123 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39123aed-e375-49b2-bfb8-97693ad8e5c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:47:36.249749 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39123aed-e375-49b2-bfb8-97693ad8e5c4 service swift] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:47:36.258573 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1470/2940] 10.222.0.64 () {60 vars in 1384 bytes} [Fri Sep 23 01:47:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 23 01:47:40.892871 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039c14c-be51-4983-aee5-b44f587cb9b3 None None] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:47:40.894496 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-4039c14c-be51-4983-aee5-b44f587cb9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:47:40.966366 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4039c14c-be51-4983-aee5-b44f587cb9b3 None None] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:47:40.971649 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969a6d7-40e0-464a-be02-16d6524d059e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:47:40.972289 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969a6d7-40e0-464a-be02-16d6524d059e service placement] REQUEST_METHOD: `GET` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:47:40.972581 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969a6d7-40e0-464a-be02-16d6524d059e service placement] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:47:40.972850 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d969a6d7-40e0-464a-be02-16d6524d059e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:47:40.974634 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-d969a6d7-40e0-464a-be02-16d6524d059e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:47:40.987403 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969a6d7-40e0-464a-be02-16d6524d059e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:47:40.989024 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d969a6d7-40e0-464a-be02-16d6524d059e service placement] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:47:40.999799 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1471/2941] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:47:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:47:44.654439 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39123aed-e375-49b2-bfb8-97693ad8e5c4 service swift] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:47:44.656142 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-39123aed-e375-49b2-bfb8-97693ad8e5c4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:47:44.724144 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39123aed-e375-49b2-bfb8-97693ad8e5c4 service swift] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:47:44.729083 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:47:44.729715 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:47:44.729997 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:47:44.730276 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] PATH_INFO: `/v3/users/0857c0af9c944a2fb6e75c7297945fc0` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:47:44.731467 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0857c0af9c944a2fb6e75c7297945fc0)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:47:44.733430 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:47:44.876506 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0857c0af9c944a2fb6e75c7297945fc0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:44.877215 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0857c0af9c944a2fb6e75c7297945fc0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:44.877840 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0857c0af9c944a2fb6e75c7297945fc0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:44.885168 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0857c0af9c944a2fb6e75c7297945fc0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:44.898668 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0857c0af9c944a2fb6e75c7297945fc0'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:44.967146 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1471/2942] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:47:44 2022] DELETE /identity/v3/users/0857c0af9c944a2fb6e75c7297945fc0 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:47:45.470281 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969a6d7-40e0-464a-be02-16d6524d059e service placement] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:47:45.483082 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d969a6d7-40e0-464a-be02-16d6524d059e service placement] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:47:45.545253 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:47:45.545916 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:47:45.546207 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:47:45.546483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] PATH_INFO: `/v3/projects/403b0f1ad71c459581759568e4fcd1ca` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:47:45.547660 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=403b0f1ad71c459581759568e4fcd1ca)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:47:45.549593 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:47:45.677040 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:47:45.694445 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '403b0f1ad71c459581759568e4fcd1ca', 'request_id': 'req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:45.700072 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '403b0f1ad71c459581759568e4fcd1ca', 'request_id': 'req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:45.793364 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '403b0f1ad71c459581759568e4fcd1ca', 'request_id': 'req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:45.806363 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] The token cache is being invalidate because project 403b0f1ad71c459581759568e4fcd1ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:47:45.806656 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:45.807337 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:45.808989 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1472/2943] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:47:45 2022] DELETE /identity/v3/projects/403b0f1ad71c459581759568e4fcd1ca => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:47:45.813492 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:47:45.815476 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:47:45.828542 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac29d854-f5c6-41a2-ad1b-f0ceb1b8545d admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:47:45.833795 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:47:45.893194 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:47:45.893841 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:47:45.894117 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:47:45.894391 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] PATH_INFO: `/v3/users/5da884d486704d019a23d46916723534` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:47:45.895598 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da884d486704d019a23d46916723534)` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:47:45.897517 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:47:46.069198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da884d486704d019a23d46916723534'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:46.069198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da884d486704d019a23d46916723534'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:46.069198 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da884d486704d019a23d46916723534'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:46.077813 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da884d486704d019a23d46916723534'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:46.091859 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.notifications [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da884d486704d019a23d46916723534'} {{(pid=81960) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:46.176941 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1472/2944] 10.222.0.64 () {58 vars in 1224 bytes} [Fri Sep 23 01:47:45 2022] DELETE /identity/v3/users/5da884d486704d019a23d46916723534 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:47:46.441479 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] Authenticating user token {{(pid=81961) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:47:46.442977 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.fernet_utils [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81961) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:47:46.457247 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b8a2d-3cff-447e-98e1-e8a6093729c9 admin admin] Validating token access rules against request {{(pid=81961) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:47:46.516483 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efae8da899ef465d9872282b2bb91fef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4e8ab21bdd4e5ebc7ac91f1638bec3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81961) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:47:46.517279 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:47:46.517390 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:47:46.517648 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] PATH_INFO: `/v3/projects/8710c6faa90b498fb6e3bae1a0df6853` {{(pid=81961) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:47:46.518804 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8710c6faa90b498fb6e3bae1a0df6853)` {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:47:46.520783 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] RBAC: Authorization granted {{(pid=81961) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:47:46.622488 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8710c6faa90b498fb6e3bae1a0df6853', 'request_id': 'req-8ace08e6-3c6a-4bcd-8552-294113be66bc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:46.628171 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8710c6faa90b498fb6e3bae1a0df6853', 'request_id': 'req-8ace08e6-3c6a-4bcd-8552-294113be66bc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:46.696346 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8710c6faa90b498fb6e3bae1a0df6853', 'request_id': 'req-8ace08e6-3c6a-4bcd-8552-294113be66bc'} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:46.708826 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] The token cache is being invalidate because project 8710c6faa90b498fb6e3bae1a0df6853 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81961) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 23 01:47:46.713415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:46.713415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG keystone.notifications [None req-8ace08e6-3c6a-4bcd-8552-294113be66bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81961) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 23 01:47:46.713415 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: [pid: 81961|app: 0|req: 1473/2945] 10.222.0.64 () {58 vars in 1233 bytes} [Fri Sep 23 01:47:46 2022] DELETE /identity/v3/projects/8710c6faa90b498fb6e3bae1a0df6853 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 23 01:47:56.172805 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:47:56.705859 dv-cinder-849143-7-iscsi devstack@keystone.service[81961]: DEBUG dbcounter [-] [81961] Writing DB stats keystone:SELECT=65,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81961) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 23 01:48:07.525649 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] Authenticating user token {{(pid=81960) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 23 01:48:07.526160 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:48:07.620292 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fa362-4112-49b0-bb42-225e7299ab94 admin admin] Validating token access rules against request {{(pid=81960) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 23 01:48:07.681257 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d58e4-e672-4509-9286-bec793947e20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f73bca3896440258430cfce042faa42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29f2f05139514a648667c0846ff3d7b7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81960) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 23 01:48:07.681873 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d58e4-e672-4509-9286-bec793947e20 service placement] REQUEST_METHOD: `GET` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 23 01:48:07.682158 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d58e4-e672-4509-9286-bec793947e20 service placement] SCRIPT_NAME: `/identity` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 23 01:48:07.682429 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d58e4-e672-4509-9286-bec793947e20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81960) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 23 01:48:07.684208 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.fernet_utils [None req-eb8d58e4-e672-4509-9286-bec793947e20 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81960) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 23 01:48:07.695508 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d58e4-e672-4509-9286-bec793947e20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 23 01:48:07.697111 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8d58e4-e672-4509-9286-bec793947e20 service placement] RBAC: Authorization granted {{(pid=81960) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 23 01:48:07.709304 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: [pid: 81960|app: 0|req: 1473/2946] 10.222.0.64 () {60 vars in 1365 bytes} [Fri Sep 23 01:48:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 23 01:48:17.703646 dv-cinder-849143-7-iscsi devstack@keystone.service[81960]: DEBUG dbcounter [-] [81960] Writing DB stats keystone:SELECT=67 {{(pid=81960) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}